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Abstract 

Integrated  System  Health  Management  (ISHM)  for  Unmanned  Aerial  Systems  (UAS)  has 
been  a  new  area  of  research  -  seeking  to  provide  situational  awareness  to  mission  and 
maintenance  operations,  and  for  improved  decision-making  with  increased  self¬ 
autonomy.  This  research  effort  developed  an  analytic  architecture  and  an  associated 
discrete-event  simulation  using  Arena  ®  to  investigate  the  potential  benefits  of  ISHM 
implementation  onboard  an  UAS.  The  objective  of  this  research  is  two-fold:  firstly,  to 
achieve  continued  airworthiness  by  investigating  the  potential  extension  of  UAS  expected 
lifetime  through  ISHM  implementation,  and  secondly,  to  reduce  life  cycle  costs  by 
implementing  a  Condition-Based  Maintenance  (CBM)  policy  with  better  failure 
predictions  made  possible  with  ISHM.  Through  a  series  of  design  experiments,  it  was 
shown  that  ISHM  presented  the  most  cost-effective  improvement  over  baseline  systems 
in  situations  where  the  reliability  of  the  UAS  is  poor  (relative  to  manned  systems)  and  the 
baseline  sensor  exhibited  poor  qualities  in  terms  of  missed  detection  and  false  alarm 
rates.  From  the  simulation  results  of  the  test  scenarios,  it  was  observed  that  failure 
occurrence  rates,  sensor  quality  characteristics  and  ISHM  performance  specifications 
were  significant  factors  in  determining  the  output  responses  of  the  model.  The  desired 
outcome  of  this  research  seeks  to  provide  potential  designers  with  top-level  perfonnance 
specifications  of  an  ISHM  system  based  on  specified  airworthiness  and  maintenance 
requirements  for  the  envisaged  ISHM-enabled  UAS. 
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I.  Introduction 


1.1  Background 

The  challenges  of  tomorrow’s  battlefield  involve  time-critical  decision  making  in  a 
massive  whirlpool  of  available  infonnation  -  and  the  best  decision  needs  to  be  made 
every  time,  all  the  time.  The  Office  of  the  Chief  Scientist  of  the  United  States  Air  Force 
(AF/ST)  released  a  report  in  May  2010  that  advocated  greater  use  of  highly  flexible 
autonomous  systems;  seeking  to  provide  significant  time-domain  operational  advantages 
over  adversaries  limited  by  human  processing  and  decision  speeds.  In  order  to  achieve 
these  gains  from  the  use  of  autonomous  systems,  new  methods  will  need  to  be  developed 
to  establish  “certifiable  trust  in  autonomy”  through  verification  and  validation  of  the 
near-infinite  state  systems  that  result  from  high  levels  of  adaptability  [Dahm,  2010]. 

In  the  domain  of  unmanned  aerial  systems  (UAS),  trust  in  autonomy  can  be  determined 
by  airworthiness,  the  safety  record,  or  the  number  of  successful  missions.  Existing 
technology  has  not  brought  UAS  to  a  state  of  complete  autonomy  with  command  and 
control  (C2)  still  residing  in  the  human  operator  during  critical  phases  of  flight  or 
mission.  Diagnostic  and  prognostic  algorithms  seek  to  improve  the  self-autonomy  of 
UAS  through  detection  and  isolation  of  faults,  and  determination  of  the  best  course  of 
action.  Although  in-flight  fault-monitoring  or  detection  protocols  currently  exist  for 
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specific  flight  critical  sub-systems,  a  UAS-wide  health  monitoring  and  decision  system 
has  seldom  been  implemented. 

For  manned  aircraft,  Health  and  Usage  Monitoring  Systems  (HUMS)  technology  had 
been  developed  since  the  1980s  in  response  to  airworthiness  concerns  for  helicopters. 
HUMS  implementation  normally  involves  a  comprehensive  suite  of  sensors  measuring 
vital  aircraft  parameters  (e.g.  vibration  and  temperature)  spanning  over  critical  sub¬ 
systems  such  as  the  engine,  rotor  and  gearbox.  The  HUMS  will  also  include  software  to 
handle  data  processing  (diagnostics)  and  prognostics  to  enhance  overall  aircraft  safety 
and  reliability  through  condition-based  maintenance  [Miller  et  ah,  1991]. 

With  the  same  motivation,  Integrated  System  Health  Management  (ISHM)  for  UAS  has 
been  a  new  area  of  research  -  seeking  to  provide  situational  awareness  to  mission  and 
maintenance  operations,  and  for  improved  decision-making  with  increased  self¬ 
autonomy.  Through  a  centralized  health  management  system,  ISHM  identifies  necessary 
sources  of  input  data  from  multiple  sensors,  generates  the  status  of  real-time  UAS 
capabilities,  and  initiates  the  best  course  of  action  in  relation  to  airworthiness  and/or 
mission  objectives.  These  objectives  can  be  measured  by  an  improvement  in  expected 
lifetime  or  an  overall  reduction  in  the  maintenance  costs. 

1.2  Problem  Statement 

According  to  the  Air  Force  Policy  Directive  62-6,  airworthiness  is  defined  as  “the 
verified  and  documented  capability  of  an  air  system  configuration  to  safely  attain, 
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sustain,  and  terminate  flight  in  accordance  with  approved  usage  and  limits”  [Donley, 
2010].  To  this  end,  Failure  Modes,  Effects  and  Criticality  Analysis  (FMECA)  has  been  a 
necessary  process  in  most  airworthiness  type  certification  plans.  FMECA  is  a  procedure 
that  identifies  potential  failure  modes  (and  effects)  of  a  system,  and  thereafter  ranks  these 
failure  modes  based  on  the  combination  of  their  severity  and  probability  of  occurrence. 
[DoD,  2005].  Although  FMECA  has  been  widely  applied  (or  mandatory  in  some  cases) 
during  conceptual  or  preliminary  design  phases,  its  applications  during  the  operational 
phase  of  a  UAS  has  been  limited.  Furthermore,  the  integration  of  FMECA  considerations 
within  an  ISHM  architecture  deserves  deeper  exploration. 

Previous  research  on  ISHM  evaluated  the  effect  of  ISHM  on  mission  effectiveness;  and  a 
baseline  model  had  been  implemented  to  quantify  the  mission-related  benefits  of  ISHM 
by  constructing  architecture  for  analysis  to  compare  against  current  autonomous  vehicle 
capabilities  [Storm,  2013].  The  focus  of  this  research  extends  beyond  previous  efforts 
through  increased  realism  of  the  ISHM  architecture  by  considering  FMECA  data  for 
UAS  airworthiness,  sensor  fusion  of  existing  and  ISHM  sensors,  and  attempts  to  develop 
a  business  case  for  condition-based  maintenance  with  improved  diagnostics  and 
prognostics  provided  by  ISHM. 

1.3  Research  Objectives  and  Hypothesis 

The  objectives  of  this  research  are  to  quantify  the  continued  airworthiness  benefits  of 
ISHM  by  developing  an  analytic  architecture  for  comparison  between  an  “as-is”  UAS 
(without  ISHM  capabilities)  and  a  “to-be”  UAS  (with  ISHM  capabilities).  From  an 
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airworthiness  certification  perspective,  this  research  aims  to  develop  a  method  for 
establishing  perfonnance  requirements  for  components  of  an  ISHM-enabled  UAS.  An 
analysis  of  the  architecture  will  examine  the  effects  of  ISHM  decisions  through  stipulated 
algorithms,  ISHM  reliability  through  the  performance  and  degradation  of  its  sensors- 
diagnostics-prognostics  suite,  and  the  associated  costs  of  maintenance. 

Adopting  a  modeling  and  simulation  approach,  the  research  presented  in  this  thesis  shall 
attempt  to  answer  the  following  questions: 

(1)  What  are  the  perfonnance  characteristics  of  ISHM  to  ensure  continued 
airworthiness  of  the  UAS? 

(2)  How  will  ISHM  provide  a  business  case  to  improve  the  level  of  UAS  self¬ 
autonomy? 

(3)  What  are  the  potential  impacts  of  ISHM  to  maintenance  practices  and  life 
cycle  costs? 

Prior  to  the  formal  research  work,  a  literature  review  was  conducted  to  answer  related 
questions  in  the  research  field: 

(1)  What  is  system  health  monitoring/management  and  what  are  some  related 
applications? 

(2)  What  are  the  essential  elements  of  ISHM? 

(3)  What  are  the  critical  FMECA  hazards  associated  with  a  typical  UAS  and  their 
relation  to  airworthiness? 

(4)  What  is  Condition-Based  Maintenance  (CBM)? 
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1.4  Methodology 

An  analytical  architecture  shall  be  developed  in  accordance  with  the  Department  of 


Defense  Architecture  Framework  [DoD,  2012],  simulating  ISHM  over  the  lifetime  of  the 
UAS  with  the  primary  architectural  goal  to  achieve  cost-effective  improvements  to 
airworthiness.  The  architecture  will  attempt  to  model  the  typical  failure  modes  of  a 
generic  UAS  and  derive  its  life  expectancy  as  a  proxy  for  continued  airworthiness 
certification  of  a  UAS,  both  with  and  without  ISHM  for  statistical  comparison.  It  is 
anticipated  that  there  will  be  a  system  design  paradigm  shift  with  this  architecture  being 
able  to  establish  performance  requirements  for  components  of  an  ISHM-enabled  UAS; 
possibly  with  reduced  redundancy  and  cost  with  information  fusion  enabled  by  ISHM  - 
to  achieve  the  same  or  higher  airworthiness  standards. 

Implementation-wise,  the  architecture  developed  shall  be  generic  enough  to  be  applied 
across  various  UAS  platfonns  with  appropriate  FMECA  information.  A  discrete-event 
simulation  approach  shall  be  adopted  to  develop  realistic  models  for  (a)  on-board  ISHM 
implementation  with  diagnostics  and/or  prognostics  algorithms,  and  (b)  purely  baseline 
sensors  available  in  UAS  hardware  without  ISHM.  As  a  secondary  objective,  a  life-cycle 
cost  model  of  the  ISHM  suite  can  also  be  derived  to  account  for  possible  degradation 
(made  apparent  through  high  false  alarm  or  missed  detection  rates)  that  will  require 
replacement  -  and  provide  a  holistic  picture  of  the  maintenance  costs  of  an  ISHM- 
enabled  UAS. 
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1.5  Assumptions  and  Limitations 

The  primary  research  objective  is  to  investigate  the  potential  benefits  of  ISHM  onboard 
UAS  -  a  relatively  new  knowledge  domain  with  limited  implementation  data.  As  such, 
there  need  to  be  several  assumptions  providing  boundaries  of  this  research  for  it  to  be 
useful: 

(1)  Without  actual  FMECA  data,  it  will  not  be  possible  to  model  and/or  evaluate 
all  the  failure  modes  for  a  typical  UAS.  As  such,  only  selected  critical  sub-system 
failures  affecting  airworthiness  of  the  UAS  shall  be  evaluated  using  the  model. 

(2)  Without  actual  sensor  perfonnance  data,  theoretical/nominal  thresholds  will 
need  to  be  assumed  to  model  sensor  degradation  in  terms  of  False  Alarm  Rate 
(FAR)  and  missed  detections. 

1.6  Implications 

From  an  airworthiness  perspective,  the  direct  implication  of  this  research  lies  in  its  ability 
to  establish  meaningful  metrics  and  design- to  requirements  for  an  ISHM-enabled  UAS.  A 
validated  ISHM  can  provide  the  desired  level  of  UAS  self-autonomy  to  detect,  diagnose 
and  implement  corrective  actions  as  necessary.  This,  in  turn,  drives  future  UAS  designs 
that  can  operate  with  reduced  redundancy  (and  maintenance  demands)  with  diagnostic 
and  prognostic  capabilities  provided  by  ISHM.  In  the  longer  term,  sustained 
airworthiness  records  with  onboard  ISHM  will  be  able  to  foster  greater  trust  in 
autonomy. 
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With  the  improved  intelligence  behind  airworthy  UAS,  complex  mission  tasks  can  then 
be  entrusted  upon  UAS  with  greater  collaborative  capabilities  (including  mission  re¬ 
planning  based  on  system  health)  and  wider  operational  envelopes  [MacConnell,  2006]. 
On  the  other  hand,  drawing  negative  parallels  from  science  fiction,  the  movie  “Oblivion” 
featured  fully  autonomous  drones  controlled  by  an  alien  artificial  intelligence  that  had 
invaded  Earth  [Kosinski,  2013].  These  drones  would  operate  collaboratively  to  dominate 
Earth  and  were  programmed  to  kill  humans  on  sight.  Therefore,  a  delicate  balance  of 
autonomy  and  delegation  of  authority  needs  to  be  established  with  ISHM.  See  Figure  1. 


Desired  ISHM  Capabilities 

Sustained  Airworthiness 
Availability 
Mission  Success 
Collaborative  Capabilities 
Reduced  Redundancy 
Lower  Costs  of  Maintenance 


Figure  1  -  Desired  Capabilities  of  ISHM  and  Unknown  Threats  of  Autonomy  [Kosinski,  2013] 
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1.7  Preview 


This  thesis  is  organized  into  five  chapters.  The  introductory  chapter  discusses  the 
background,  problem  statement  and  objectives  of  the  research.  The  descriptions  of  the 
ensuing  chapters  are  as  follow: 

•  Chapter  II  examines  the  current  state  of  system  health 
monitoring/management  and  its  related  applications,  and  provides  an 
understanding  of  the  main  elements  of  ISHM.  This  chapter  also  highlights 
airworthiness  and  system  safety  concepts,  presents  a  preliminary  FMECA 
hazard  assessment  of  a  typical  UAS,  and  discusses  the  benefits  of  the  ideal 
ISHM  system. 

•  Chapter  III  describes  the  research  methodology  through  the  proposed 
analytic  architecture. 

•  Chapter  IV  presents  the  results  and  associated  analysis  of  the  discrete- 
event  simulation  model. 

•  Chapter  V  draws  conclusions  regarding  the  research  objectives,  and 
proposes  potential  areas  for  future  research. 
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II.  Literature  Review 


2.1  Chapter  Overview 

The  objectives  of  this  chapter  are  to  examine  the  current  state  of  system  health 
monitoring/management  and  its  related  applications,  provide  an  understanding  of  the 
main  elements  of  ISHM,  highlight  airworthiness  and  system  safety  concepts,  present  a 
preliminary  FMECA  hazard  assessment  of  a  typical  UAS,  and  discuss  the  benefits  of  the 
ideal  ISHM  system. 

2.2  System  Health  Management 

This  section  presents  various  definitions  of  system  health  monitoring/management  in  its 
various  related  applications. 

2.2.1  Application  in  Health  Care 

There  exist  varied  definitions  and  applications  of  system  health  management;  with  its 
most  direct  application  in  health  care.  In  a  recent  technology  ‘disruption’  to  health  care, 
IBM’s  Watson  -  the  same  machine  that  beat  Ken  Jennings  at  Jeopardy  -  was  being 
‘tutored’  at  Memorial  Sloan-Kettering,  perusing  through  medical  case  histories  and 
learning  to  make  diagnoses  and  treatment  recommendations  [Cohn,  2013].  This 
innovation  seeks  intelligence  beyond  simple  electronic  look-ups  of  medical  encyclopedia. 
Although  the  future  of  a  robot  seeing  a  patient  in  place  of  a  human  doctor  remains  to  be 
seen,  intelligence  in  the  form  of  Watson  provides  consistency  of  decisions  amongst 
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available  medical  solutions,  based  on  accurate  clinical  examinations  and  evidence. 
Another  medical  innovation  exists  in  the  form  of  a  device  called  the  Stealth  Vest, 
wearable  sensor  technology  that  can  continuously  communicate  data  without  the  patient 
even  being  aware  of  it  [Glen,  2012].  This  was  developed  by  a  group  of  researchers  based 
at  Emory  University  and  Georgia  Tech  primarily  for  teenagers,  who  are  less  likely  to 
comply  with  physician  instructions  about  taking  readings  or  medications. 

The  technological  breakthroughs  in  health  care,  in  the  areas  of  sensor  data  fusion  and 
artificial  intelligence,  presented  similarities  to  an  ideal  ISHM  architecture  that  integrates 
processed  sensor  information  and  intelligence  through  diagnostics  and  prognostics 
algorithms. 

2.2.2  Health  and  Usage  Monitoring  System  (HUMS)  in  Helicopters 

Increased  demand  for  improved  operational  safety  and  reduced  rotorcraft  maintenance 
costs  had  paved  the  way  for  HUMS  [Wiig,  2006].  These  systems  emerged  in  the  1980s  as 
a  response  to  the  high  accident  rates  experienced  by  offshore  shuttle  helicopters 
traversing  the  petrol  installations  in  the  North  Sea.  The  UK  Civil  Aviation  Authority 
(CAA)  defined  HUMS  in  two  main  subsystems:  a  Vibration  Monitoring  System  (VMS), 
and  a  Usage  Monitoring  System  (UMS).  The  latter  included  functions  such  as 
temperature  and  torque  monitoring,  magnetic  plugs  and  chip  detectors.  The  VMS 
addresses  the  Health  aspect  of  HUMS  and  should  monitor: 

•  engine  to  main  gearbox  input  drive  shafts, 
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•  main  gearbox  shafts,  gears  and  bearings, 

•  accessory  gears,  shafts  and  bearings, 

•  tail  rotor  drive  shafts  and  bearings, 

•  intermediate  and  tail  gearbox  gears,  shafts  and  bearings, 

•  oil  cooler  drive,  and 

•  main  and  tail  rotor  track  and  balance. 

In  terms  of  process  management,  the  typical  processes  of  a  HUMS  program  are  depicted 
in  Figure  2  [JHSIT,  2013].  Basic  HUMS  operation  requires  that  data  be  displayed  on  a 
ground  station  after  download,  identifying  any  primary  indicators  exceeding  their  pre¬ 
defined  thresholds.  Recent  history  of  primary  and  secondary  data  should  also  be  available 
to  maintenance  personnel  for  comparison  against  past  alerts  (or  false  alarms)  to  ensure 
continued  airworthiness  of  the  aircraft.  The  ability  to  trend  data  and  facilitate  comparison 
with  other  aircraft,  fleet  average  thresholds  or  other  health  indicators  is  also 
recommended. 


Figure  2  -  HUMS  Processes  [JHSIT,  2013] 
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The  following  processes  are  defined  for  a  HUMS  program  [JHSIT,  2013]: 


•  Data  Acquisition  and  Transfer  -  The  process  of  acquiring  data  from  the  various 
sensors  and  transferring  it  to  a  ground  station.  It  is  important  to  download  HUMS 
data  regularly  to  ensure  currency  of  HUMS  data. 

•  Data  Analysis  -  This  includes  the  review  of  HUMS  data  by  a  maintainer  on  the 
flight  line  for  advisories  or  threshold  exceedances,  followed  by  a  detailed  analysis 
by  a  trained  HUMS  analyst  or  engineer.  The  latter  is  best  accomplished  by 
trending  historical  vibration  data  against  the  rest  of  the  fleet  -  and  this  helps  in  the 
analysis  for  lesser  known  faults. 

•  Data  Validation  -  Whenever  a  measurement  is  recorded,  there  is  a  chance  for 
error.  As  such,  whenever  a  HUMS  generates  an  alert,  an  effort  must  be  made  to 
ensure  that  the  alert  is  valid.  A  collaborative  data  exchange  utilizing  maintenance 
records,  pilot  and  mechanic  recorded  discrepancies,  vibration  readings,  oil 
analysis,  visual  inspection  etc.  will  be  essential  in  reducing  such  erroneous  alerts 
or  false  alanns. 

•  Training  -  It  is  important  that  technicians  or  maintainers  are  adequately  trained  to 
deliver  the  first  level  of  analysis  on  the  flight  line  -  critical  for  releasing  an 
aircraft  for  subsequent  flights.  Thereafter,  HUMS  analysts  or  engineers  will  then 
need  to  be  provided  additional  tools  to  deliver  deeper  analysis  and  trending 
information. 


HUMS  originated  from  an  airworthiness  concern,  and  its  implementation  drove  extensive 
research  in  condition-based  rotorcraft  maintenance.  Understanding  the  HUMS 
architecture  and  associated  processes  will  provide  alignment  in  this  research  in 
developing  maintenance  cost  models. 


2.2.3  Integrated  Vehicle  Health  Management 

Integrated  vehicle  health  management  (IVHM)  is  a  collection  of  data  relevant  to  the 
present  and  future  perfonnance  of  a  vehicle  system  and  its  transfonnation  into 


information  can  be  used  to  support  operational  decisions.  This  design  and  operation 
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concept  embraces  an  integration  of  sensors,  communication  technologies,  and  artificial 
intelligence  to  provide  vehicle-wide  abilities  to  diagnose  problems  and  recommend 
solutions  [Benedettini  et  al.,  2009].  The  author  also  presented  various  definitions  of 
IVHM  found  in  the  literature,  as  presented  in  Table  1. 


Table  1  -  IHVM  Definitions  |Benedettini  et  al.,  2009] 


Author 

Definition  of  IVHM 

NASA,  1992 

‘The  capability  to  efficiently  perform  checkout,  testing,  and  monitoring  of 
space  transportation  vehicles,  subsystems,  and  components  before,  during, 
and  after  operation.’... ‘must  support  fault-tolerant  response  including 
system/subsystem  reconfiguration  to  prevent  catastrophic  failure;  and 
IVHM  must  support  the  planning  and  scheduling  of  post-operational 
maintenance.’ 

Aaseng,  2001 

‘All  the  activities  that  are  performed  to  understand  the  state  of  the  vehicle 
and  its  components,  to  restore  the  vehicle  to  nominal  system  status  when 
malfunctions  occur,  and  to  minimize  safety  risks  and  mission  impacts  that 
result  from  system  failures’ 

Baroth  et  al.,  2001 

An  ‘effort  to  coordinate,  integrate,  and  apply  advanced  software,  sensors, 
and  design  technologies  to  increase  the  level  of  intelligence,  autonomy, 
and  health  state  determination  and  response  of  future  vehicles’ 

Roemer  et  al.,  2001 

‘Integrates  component,  subsystem,  and  system  level  health  monitoring 
strategies,  consisting  of  anomaly/diagnostic/prognostic  technologies,  with 
an  integrated  modelling  architecture  that  addresses  failure  mode 
mitigation  and  lifecycle  costs’ 

Price  et  al.,  2003 

‘An  example  of  an  intelligence  sensing  system.  The  purpose  of  such 
system  is  to  detect  and  measure  certain  quantities,  and  to  use  the 
information  and  knowledge  obtained  from  the  measured  data,  and  any 
prior  knowledge,  to  make  intelligent,  forward-looking  decisions,  and 
initiate  actions’ 

Wilmering,  2003 

‘The  unified  capability  of  an  arbitrarily  complex  system  of  systems  to 
accurately  assess  the  current  state  of  member  system  health,  predict  some 
future  state  of  the  health  of  member  systems,  and  assess  that  state  of 
health  within  the  appropriate  framework  of  available  resources  and 
operational  demand’ 

Paris  et  al.,  2005 

‘The  process  of  assessing,  preserving,  and  restoring  system  functionality 
across  flight  and  ground  systems’ 

Jakovljevic  and  Artner,  2006 

‘Ensures  the  reliable  capture  of  the  "health  status”  of  the  overall  aerospace 
system  and  helps  to  prevent  its  degradation  or  failure  by  providing  reliable 
information  about  problems  and  faults’ 

Karsai  et  al.,  2006 

‘Its  goal  is  to  provide  better  ways  for  operating  and  maintaining  aerospace 
vehicles  using  techniques,  such  as  condition  monitoring,  anomaly 
detection,  fault  isolation,  and  managing  the  vehicle  operations  in  the  case 
of  faults’ 
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2.3  Architecture  of  ISHM 


From  the  literature  presented,  Figure  3  depicts  a  suggested  architecture  for  ISHM. 


Figure  3  -  Typical  ISHM  Architecture  [Benedettini  et  al.,  2009] 


At  the  front  end  of  the  architecture  is  a  sensor  suite  responsible  for  gathering  state 
awareness  variables  that  are  indicative  of  potential  failure  modes.  For  an  envisaged 
ISHM  configuration,  apart  from  conventional  sensors  that  monitor  and  control  sub¬ 
systems,  system-level  sensor  suites  are  also  being  introduced  in  the  form  of  smart 
embedded  sensor  systems  with  wireless  communications  transfer  protocol  in  place  for 
overall  system  health  management.  Upon  filtering  of  sensor  data  to  extract  relevant  fault 
features,  the  diagnostics  module  analyzes  the  fault  features  to  detect,  identify  and  isolate 
impending  fault  conditions.  In  addition,  with  health  and  usage  data  being  fed  to  the 
prognostic  module,  the  latter  is  able  to  combine  historical  data  to  generate  an  estimation 
of  the  time-to-failure  of  specific  subsystems  and  components.  Depending  on  the  level  of 
autonomy,  such  diagnostic  and  prognostic  information  can  be  processed  on-board  the 
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vehicle  through  its  auto-recovery  systems  or  communicated  to  technical  support 
managers  on  ground. 


Although  there  exist  various  definitions  of  IHVM,  they  seem  to  align  to  this  ideal  ISHM 
configuration  -  that  will  provide  a  basis  for  the  analytic  architecture  of  this  research. 


2.4  Airworthiness  and  System  Safety  Framework 

System  safety  is  the  application  of  engineering  and  management  principles,  criteria  and 
techniques  to  optimize  all  aspects  of  safety  within  the  constraints  of  operational 
effectiveness,  time  and  cost  throughout  all  phases  of  the  system  life  cycle.  MIL-STD-882 
is  the  primary  reference  for  system  safety  program  infonnation  for  DoD  weapon  systems. 
A  system  safety  program  is  crucial  for  the  initial  and  continued  airworthiness  of  all 
weapon  systems;  with  the  following  objectives  as  listed  in  the  Air  Force  System  Safety 
Handbook  [AFSC,2000]: 


a.  Safety,  consistent  with  mission  requirements,  is  designed  into  the  system  in  a 
timely,  cost-effective  manner; 

b.  Hazards  are  identified,  evaluated,  and  eliminated,  or  the  associated  risk  reduced  to 
a  level  acceptable  to  the  managing  activity  (MA)  throughout  the  entire  life  cycle 
of  a  system; 

c.  Historical  safety  data,  including  lessons  learned  from  other  systems,  are 
considered  and  used; 

d.  Minimum  risk  is  sought  in  accepting  and  using  new  designs,  materials,  and 
production  and  test  techniques; 

e.  Actions  taken  to  eliminate  hazards  or  reduce  risk  to  a  level  acceptable  to  the  MA 
are  documented; 

f.  Retrofit  actions  are  minimized; 
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g.  Changes  in  design,  configuration,  or  mission  requirements  are  accomplished  in  a 
manner  that  maintains  a  risk  level  acceptable  to  the  MA; 

h.  Consideration  is  given  to  safety,  ease  of  disposal,  and  demilitarization  of  any 
hazardous  materials  associated  with  the  system; 

i.  Significant  safety  data  are  documented  as  “lessons  learned”  and  are  submitted  to 
data  banks,  design  handbooks,  or  specifications; 

j.  Hazards  identified  after  production  are  minimized  consistent  with  program 
constraints. 


Understanding  airworthiness  requirements  and  the  fundamental  objectives  of  the  System 
Safety  Framework  provides  the  impetus  to  this  research. 

2.4.1  Mishap  Severity  and  Probabilities 

In  accordance  with  the  system  safety  framework  and  MIL-STD-882,  hazard  analyses 
based  on  failure  modes  are  classified  according  to  mishap  severity  categories  and 
probabilities  listed  in  Table  2.  Appropriate  risk  mitigation  measures  are  devised  after  the 
risk  assessment  has  been  made.  The  mishap  assessment  also  serves  as  a  guideline  for  the 
appropriate  authorities  to  accept  any  residual  risk  after  mitigation  measures  have  been 
implemented. 

For  this  research,  different  severity  categories  and  probability  levels  of  failure  modes 
will  initiate  different  courses  of  actions  in  the  discrete-event  simulation. 
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Table  2  -  Suggested  Mishap  Severity  Categories  and  Probability  Levels  |DoD,  2000] 


Description 

Category 

Environmental,  Safety,  and  Health  Result  Criteria 

Catastrophic 

I 

Could  result  in  death,  permanent  total  disability,  loss 
exceeding  $1M,  or  irreversible  severe  environmental 
damage  that  violates  law  or  regulation. 

Critical 

II 

Could  result  in  permanent  partial  disability,  injuries 
or  occupational  illness  that  may  result  in 
hospitalization  of  at  least  three  personnel,  loss 
exceeding  $200K  but  less  than  $1M,  or  reversible 
environmental  damage  causing  a  violation  of  law  or 
regulation. 

Marginal 

III 

Could  result  in  injury  or  occupational  illness 
resulting  in  one  or  more  lost  work  days(s),  loss 
exceeding  $10K  but  less  than  S200K,  or  mitigatible 
environmental  damage  without  violation  of  law  or 
regulation  where  restoration  activities  can  be 
accomplished. 

Negligible 

IV 

Could  result  in  injury  or  illness  not  resulting  in  a  lost 
work  day,  loss  exceeding  $2K  but  less  than  $10K,  or 
minimal  environmental  damage  not  violating  law  or 
regulation. 

Description* 

Level 

Specific  Individual  Item 

Fleet  or  Inventory** 

Frequent 

A 

Likely  to  occur  often  in  the 
life  of  an  item,  with  a 
probability  of  occurrence 
greater  than  10'1  in  that  life. 

Continuously 

experienced. 

Probable 

B 

Will  occur  several  times  in  the 
life  of  an  item,  with  a 
probability  of  occurrence  less 
than  10'1  but  greater  than  10'2 
in  that  life. 

Will  occur  frequently. 

Occasional 

C 

Likely  to  occur  some  time  in 
the  life  of  an  item,  with  a 
probability  of  occurrence  less 
than  10'2  but  greater  than  1  O'3 
in  that  life. 

Will  occur  several 
times. 

Remote 

D 

Unlikely  but  possible  to  occur 
in  the  life  of  an  item,  with  a 
probability  of  occurrence  less 
than  10'3  but  greater  than  1  O'6 
in  that  life. 

Unlikely,  but  can 
reasonably  be 
expected  to  occur. 

Improbable 

E 

So  unlikely,  it  can  be  assumed 
occurrence  may  not  be 
experienced,  with  a 
probability  of  occurrence  less 
than  10-6  in  that  life. 

Unlikely  to  occur,  but 
possible. 
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2.4.2  Hazard  Reduction  Precedence 


In  line  with  the  system  safety  program  objectives,  the  order  of  precedence  for  satisfying 
system  safety  requirements  and  resolving  identified  hazards  is  depicted  in  Figure  4 
[AFSC,  2000]. 

Step  1.  Design  for  Minimum  Risk  -  Design  to  eliminate  hazards.  If  an  identified 
hazard  cannot  be  eliminated,  reduce  the  associated  risk  to  an  acceptable  level,  as 
defined  by  the  MA,  through  design  selection. 

Step  2.  Incorporate  Safety  Devices  -  If  identified  hazards  cannot  be  eliminated  or 
their  associated  risk  adequately  reduced  through  design  selection,  that  risk  shall  be 
reduced  to  a  level  acceptable  to  the  MA  through  the  use  of  fixed,  automatic,  or 
other  protective  safety  design  features  or  devices. 

Step  3.  Provide  Warning  Devices  -  When  neither  design  nor  safety  devices  can 
effectively  eliminate  identified  hazards  or  adequately  reduce  associated  risk, 
device  shall  be  used  to  detect  the  condition  and  to  produce  an  adequate  warning 
signal  to  alert  personnel  of  the  hazard. 

Step  4.  Develop  Procedures  and  Training  -  Where  it  is  impractical  to  eliminate 

hazards  through  design  selection  or  adequately  reduce  the  associated  risk  with 

safety  and  warning  devices,  procedures  and  training  shall  be  used.  Procedures 

may  include  the  use  of  personal  protective  equipment. 
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With  an  ISHM-enabled  UAS,  it  is  anticipated  that  there  may  be  a  paradigm  shift  in  UAS 
design  in  terms  of  hazard  reduction  precedence.  Through  better  sensor  data  fusion,  less 
effort  may  be  focused  on  developing  redundan  t  designs  (Step  1)  or  safety  devices  (Step 
2).  Instead,  better  diagnostics  or  prognostics  algorithms  may  be  developed  to  ensure  that 
the  UAS  will  always  be  able  to  detect  (or  even  predict)  a  fault  condition  and  execute  safe 
recovery  actions. 


Figure  4  -  Hazard  Reduction  Precedence  [AFSC,  2000| 


2.4.3  Unmanned  Aerial  Systems  Safety  by  Mass  Classification  -  Ground  Impact 
Hazard  Analysis 

The  severity  definitions  (refer  to  Table  2)  related  to  occupants  of  the  aircraft  do  not  apply 
to  an  unmanned  system.  In  UAS  operation,  the  most  severe  possible  outcomes  are  those 
that  result  in  injury  to  the  general  public,  either  in  other  aircraft  or  on  the  ground.  Ground 
impact  can  endanger  the  general  public,  and  midair  collision  with  a  manned  aircraft  can 
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threaten  the  safety  of  the  passengers  aboard  that  aircraft.  Both  effects  are  critical  system 
design  drivers  that  have  implications  for  UAS  operations  and  reliability  requirement.  In  this 
section,  a  ground  impact  hazard  analysis  conducted  by  MIT  is  presented  [Weibel  and 
Hansman,  2005].  The  ground  impact  model  considers  varying  risk  of  ground  impact  across 
the  entire  CONUS  area  based  on  population  density.  An  UAS  accident  ‘exposes’  the  general 
public  to  potential  harm,  but  does  not  necessarily  directly  result  in  a  fatality.  In  simplified 
terms,  the  model  considered  an  average  area  of  exposure  for  which  the  accident  has  effects 
(which  can  be  considered  as  the  lethal  debris  area),  estimated  by  the  term  Aexp  -  which  is 
determined  by  the  UAS  class  based  on  its  frontal  area.  In  addition,  the  UAS  accident  must 
also  penetrate  sheltering,  such  as  houses  and  vehicles,  before  coming  into  contact  with 
persons.  The  proportion  of  time  that  the  debris  will  penetrate  shelter  given  exposure  is 
modeled  by  the  penetration  factor,  Ppen.  It  is  assumed  that  if  debris  penetrates  sheltering,  then 
a  fatality  has  occurred.  The  ground  impact  model  was  applied  to  six  UAS  from  the  Heavy, 
HALE  (High  Altitude  Long  Endurance),  MALE  (Medium  Altitude  Long  Endurance), 
Tactical,  Mini  and  Micro  classifications.  Table  3  summarizes  the  parameters  of  the  model. 
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Table  3  -  UAS  Classes  for  Ground  Impact  Analysis  [Weibel  and  Hansman,  2005] 


Representative  Vehicles 

Weight 

^  exp 

Estimated 

Pp  en 

Heavy 

* 

602,500  lb 

7700  ft2 

100% 

HALE 

[▼ 

L  ~*T  •©■ 

■ 

i 

25,600  lb 

900  ft2 

90% 

MALE 

2,250  lb 

360  ft2 

60% 

Tactical 

2^ 

3511b 

30  ft2 

25% 

Mini 

9.6  1b 

14  ft2 

10% 

Micro 

■ 

m 

0.14  lb 
(2.16  oz) 

0.26  ft2 

5% 

The  objective  of  the  analysis  was  to  calculate  the  target  level  of  reliability  for  each  UAV 
class  in  order  to  meet  an  assumed  target  level  of  safety  of  10'7  fatalities  per  hour  of  UAS 
operation.  The  study  concluded  that  there  is  an  increase  in  required  reliability  of  the  UAS 
as  vehicle  mass  increases.  This  implied  that  the  inherent  risk  of  operating  a  heavier  UAS 
is  higher  when  addressing  ground  impact  hazards.  Specifically  for  HALE  UAVs,  they 
would  need  to  meet  reliability  levels  of  current  manned  military  or  general  aviation  aircraft, 
on  the  order  of  100,000  hr  between  accidents,  to  overfly  20%  of  the  country  at  the  target 
level  of  safety. 
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This  study  established  a  direct  relationship  between  the  weight  of  a  UAS  and  its  inherent 
operating  risk.  In  addition,  it  suggested  a  proxy  for  this  research  in  terms  of  target  reliability 
to  define  the  airworthiness  standard  of  an  UAS. 

2.5  Failure  Mode,  Effects  and  Criticality  Analysis 

FMECA  is  a  procedure  for  identifying  potential  failure  modes  in  a  system  and  classifying  them 
according  to  their  severity.  A  FMECA  is  usually  carried  out  progressively  in  two  parts.  The  first 
part  identifies  failure  modes  and  their  effects  (also  known  as  failure  modes  and  effects  analysis). 
The  second  part  ranks  the  failure  modes  according  to  the  combination  of  their  severity  and  the 
probability  of  occurrence  (criticality  analysis)  [DoD,  2005],  A  general  approach  for  conducting  a 
FMECA  is  shown  in  Figure  5.  The  steps  listed  are  self-explanatory.  However,  a  distinction  needs 
to  be  made  between  determining  the  severity  of  a  failure  mode,  and  the  failure  mode  criticality. 
The  latter  is  a  function  of  severity,  the  frequency  of  occurrence  of  a  failure  mode,  and  the 
probability  that  it  will  be  detected  in  time  to  preclude  its  impact  at  the  system  level.  This 
criticality  analysis  resulted  in  the  determination  of  the  risk  priority  number  (RPN)  as  a  metric  for 
evaluation.  RPN  can  be  expressed  as 

RPN  =  (severity  rating)  x  (frequency  rating)  x  (probability  of  detection  rating) 

The  RPN  reflects  failure -mode  criticality;  and  on  inspection,  one  can  see  that  a  failure  mode  with 
a  high  frequency  of  occurrence,  with  significant  impact  on  system  performance,  and  that  is 
difficult  to  detect  is  likely  to  have  a  high  RPN  [Blanchard,  2004]. 

FMECA  evaluations  form  the  basis  of  the  architecture  -  and  the  analysis  results  will  determine 
the  performance  requirements  of  individual  subsystems  and  components  in  achieving  defined 
airworthiness  standards. 
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Figure  5  -  General  Approach  to  Conducting  a  FMECA  [Blanchard,  2004] 


2.5.1  Preliminary  Hazard  Assessment  of  Generic  UAS 

NASA  published  a  report  in  2007  that  identified  the  typical  failure  conditions  of  a  UAS 
based  on  functional  decomposition  of  a  generic  UAS  [Hayhurst  et  al.,  2007].  The  full 
functional  decomposition  is  relatively  large,  with  69  functions  at  the  lowest  level  under 
the  major  functions  of  aviate,  navigate,  communicate  and  mitigate.  Figure  6  shows  the 
top-level  view  of  these  functions. 


•  Aviate  includes  not  only  actions  involved  in  flying  the  aircraft,  but  also  actions  for 
moving  the  aircraft  on  the  ground,  providing  command  and  control,  and  managing 
sub-systems. 

•  Navigate  includes  actions  involved  in  the  management  and  execution  of  a  flight 
plan. 

•  Communicate  provides  functionality  for  the  communication  between  the  UAS, 
ATC  and  other  aircraft.  Ah  actions  associated  with  the  command  and  control  link 
to  the  vehicle  are  contained  within  the  Aviate  category. 
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•  Mitigate  includes  actions  such  as  avoiding  traffic,  avoiding  ground  objects, 
avoiding  weather  or  other  types  of  environmental  effects,  and  handling 
contingencies. 
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Figure  6  -  Top-Level  View  of  UAS  Functional  Decomposition  [Flayhurst  etal.,  2007] 


The  primary  safety  goal  of  the  failure  hazard  assessment  is  to  avoid  any  UAS-initiated 
decrease  in  the  safety  of  the  National  Air  Space  (NAS).  As  a  result,  failure  condition 
criticality  is  determined  by  its  effect  on  people  on  ground  or  in  other  aircraft.  The  latter 
case  includes  stress  or  injury  to  occupants  of  other  aircraft  as  a  result  of  an  evasive 
maneuver.  Damage  to  material  assets  is  out-of-scope,  unless  it  affects  human  safety.  The 
assessment  was  applied  to  69  different  functions  (that  equated  to  the  69  leaf  nodes  of  the 
functional  decomposition).  Figure  7  shows  the  total  number  of  failure  conditions  by  the 
four  major  categories  in  the  functional  decomposition.  The  majority  of  potential  failure 
conditions  fall  under  the  Aviate  or  Mitigate  functions.  Figure  8  presents  the  same  data, 
with  detail  regarding  the  number  of  failure  conditions  per  severity  level. 
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Figure  7  -  Failure  Condition  Totals  by  Functional  Category  [Flayhurst  et  al.,  2007] 


Catastrophic 

Hazardous 

Major 

Minor 

No  Safety  Effect 


Figure  8  -  Failure  Condition  Severities  by  Functional  Category  [Flayhurst  et  al.,  2007] 

The  majority  of  failure  conditions  with  catastrophic  and  hazardous  consequences  are 
found  in  the  Aviate  and  Mitigate  functions.  In  the  assessment  by  NASA,  twenty-six 
potentially  catastrophic  failure  conditions  were  identified,  considering  only  single  failures 
in  the  en-route  phase  of  flight.  An  interesting  observation  to  make  at  this  point  is  how  the 
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number  of  catastrophic  failure  conditions  for  a  generic  UAS  compares  with  those 
numbers  assumed  for  commercial  transport  aircraft  and  for  general  aviation  aircraft. 
According  to  AC  23.1309-1C1,  there  are  ten  catastrophic  failure  conditions  assumed  for  a 
general  aviation  aircraft  (covering  single  and  multiple  failures  over  all  phases  of  flight); 
and  there  are  100  catastrophic  failure  conditions  assumed  for  a  commercial  transport 
aircraft  according  to  AC  25.1309-1A".  While  recognizing  that  these  are  broad 
generalizations,  preliminary  indications  are  that  the  number  of  potential  catastrophic 
failure  conditions  for  a  generic  UAS  will  be  greater  than  the  number  for  general  aviation 
aircraft;  and  the  relation  of  the  estimate  to  commercial  transport  aircraft  will  have  to 
depend  on  further  assessment  of  failure  conditions  in  all  phases  of  flight. 

Understanding  the  functional  decomposition  of  a  generic  UAS  and  the  potential  failure 
conditions  aids  in  this  research  by  providing  the  top-level  functional  failure  modes  for 
analysis. 

2.6  Benefits  of  the  Ideal  ISHM  System 

The  preceding  section  discussed  the  development  and  applications  of  system  health 
management  in  recent  years.  At  present,  health  management  is  already  part  of  the 
standard  performance  and  maintenance  paradigms  in  the  propulsion  and  rotorcraft  arenas. 
Although  its  influence  is  steadily  growing,  health  management  (or  pure  monitoring?)  is 


1  FAA  Advisory  Circular  (AC)  23.1309-1C  -  Equipment,  Systems,  and  Installations  in  Part  23  Airplanes. 
Date  Issued:  12  Mar  1999.  Currently  superseded  by  AC  23.1309-1E  issued  on  17  Nov  2011. 

2  FAA  Advisory  Circular  (AC)  25.1309-1A  -  System  Design  and  Analysis.  Date  Issued:  21  Jun  1988. 
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still  viewed  primarily  as  a  means  for  detecting  damage  and/or  failures  in  support  of 
maintenance  activities;  with  health  management  systems  being  an  ‘after-thought’  of 
system  designs.  This  led  to  the  development  of  health  management  systems  focused  on 
individual  components  of  subsystems  with  data  generated  being  typically  viewed  only 
within  the  context  of  that  subsystem.  Yet,  it  is  a  common  belief  that  integrated  system 
health  management  offers  far  more  benefits  than  is  being  envisioned  currently 
[MacConnell,  2006]. 

2.6.1  Benefit  Classes 

In  a  collaborative  research  effort  with  the  Air  Force,  industry  and  academia,  MacConnell 
categorized  the  benefits  of  ISHM  in  the  following  four  classes  [MacConnell,  2006]. 
These  classes  presented  potential  areas  within  the  ISHM  architecture  where  Measures  of 
Performance  (MOP)  or  Measures  of  Effectiveness  (MOE)  can  be  defined. 

Mission  Availability  —  These  benefits  encompass  all  aspects  of  ISHM  that  involve 
getting  a  system  ready  for  mission  launch  from  in-flight  fault  diagnostics  triggering  a 
maintenance  action  to  ground  check-out.  Go/No-go  decision  making  based  on  knowledge 
of  remaining  time  to  failure.  This  category  includes  conventional  Condition  Based 
Maintenance  (CBM)  and  is  geared  towards  making  sure  the  vehicle  is  ready  to  perform 
its  mission  when  assigned.  These  scenarios  are  heavily  dependent  on  diagnostics, 
remaining  life  assessments  and  automation  and  communication. 
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Mission  Success  —  This  category  encompasses  all  scenarios  that  result  in  a  platfonn 
being  able  to  accomplish  its  mission  after  “launch”  regardless  of  in-flight  faults,  failures 
or  damage  sustained  during  the  mission.  This  deals  primarily  with  real-time  vehicle  state 
sensing  and  autonomous  decision-making  during  a  mission  in  particular  dealing  with  in¬ 
flight  faults/events. 

Mission  Capability  —  This  category  focuses  on  the  potential  of  new  capabilities  to 
improve  performance.  This  category  in  particular  includes  new  approaches  to  integrating 
differing  subsystems  to  generate  new  abilities.  It  also  addresses  the  development  and 
exploitation  of  theater-wide  ISHM  based  planning  and  execution. 

Design  Paradigm  —  This  category  addresses  the  indirect  benefits  or  those  benefits 
which  are  enabled  by  ISHM;  and  these  benefits  are  not  necessarily  reflected  in  the  war¬ 
fighters  experience  of  the  platform  but  rather  in  the  process  of  the  design  and 
manufacture  of  the  system.  This  benefit  class  includes  concepts  such  as  dramatically 
reduced  factors  of  safety  for  design  and  revolutionary  certification  processes. 

2.6.2  The  ISHM  Business  Case  -  Condition  Based  Maintenance 

Though  the  benefits  of  ISHM  are  well  recognized,  the  most  quantifiable  benefits  in  the 
current  paradigm  were  almost  exclusively  reported  in  terms  of  maintenance  related 
savings.  Maintenance  policies  generally  define  two  types  of  maintenance: 

•  Time  Based  Maintenance  (TBM),  also  known  as  scheduled/preventive 
maintenance  where  components  are  replaced  at  specified  intervals  to  preclude 
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failures  during  operation.  Due  to  design  especially  in  cases  where  redundancy  is 
absent,  stringent  and  conservative  replacement  intervals  are  implemented  way 
before  a  component’s  actual  failure. 

•  Condition  Based  Maintenance  (CBM),  also  known  as  unscheduled/corrective 
maintenance  where  components  are  replaced  upon  detection  of  an  unsatisfactory 
condition  -  such  as  those  detected  (or  even  predicted)  by  an  ISHM  system. 

Every  component  has  a  safe  life  or  operating  life,  beyond  which  continued  operation  of 

the  component  could  result  in  catastrophic  failures.  In  a  TBM  maintenance  policy,  there 

is  potential  for  wastage  should  a  component  be  replaced  way  ahead  of  its  life  expiration. 

With  the  potential  of  increased  health  management  and  surveillance  presented  by  ISHM, 

the  useful  life  of  a  component  can  be  extended  until  diagnostic  or  prognostic  algorithms 

decide  that  an  impending  failure  is  imminent.  The  algorithms  will  have  to  consider  the 

uncertainty  and  confidence  levels  regarding  a  component’s  useful  life  in  order  not  to 

encroach  upon  the  unsafe  operating  window.  The  concept  of  savings  through  a  CBM 

philosophy  can  be  illustrated  by  Figure  9. 

A  CBM  life  cycle  cost  model  can  be  incorporated  within  the  analytic  architecture  for  a 
cost-benefit  analysis  of  an  ISHM-enabled  UAS. 
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Figure  9  -  Illustration  of  Savings  from  Condition  Based  Maintenance 

29 


The  literature  also  presented  potential  benefits  in  the  following  areas  that  strengthen  the 
business  case  of  ISHM,  as  summarized  in  Table  4  [MacConnell,  2006]. 


Table  4  -  The  ISHM  Business  Case  [MacConnell,  2006| 


The  ISHM  Business  Case  of  Today 

Maintenance  Time  Savings 

Direct  cost/time  savings  due  to  proper  diagnosis,  reduction  in  scheduled  maintenance  and  reduction  in 
inspection  time. 

False  Alarm  Avoidance 

Direct  cost/time  savings  due  to  elimination  of  unnecessary  maintenance;  arising  from  Cannot 
Duplicate  (CND)  and  Retest  OK  (RTOK)  cases. 

Availability  Improvement 

Direct  benefit  due  to  downtime  elimination. 

Spares  and  Supply  Savings 

Direct  cost/time  savings  due  to  knowing  what  is  needed  and  when  it  is  needed. 

Recurring  Cost  Savings 

Strict  cost  savings  due  to  health  management. 

The  ISHM  Business  Case  of  Tomorrow 

A  fully  ISHM  enabled  system  will  cost  only  70%  of  a  comparable  conventional  system  for  equivalent 
capability  \ 

50%  reduction  in  support  costs  due  to  false  alarm,  CND  and  RTOK  elimination,  minimized  diagnostic 
time  and  reduction  of  unnecessary  maintenance. 

10%  reduction  in  Acquisition  costs  due  to  reduced  conservatism  in  design  due  to  relaxed  safety 
margins  with  increased  real-time  status  knowledge,  and  reduced  redundancy  resulting  in  reduced 
weight. 

4%  reduction  in  required  fleet  size  based  on  ability  to  do  more  with  less  due  to  increased  availability, 
reduced  abort  and  cancellation  rates,  and  increased  endurance  and  range. 

A  fully  ISHM  enabled  design  will  be  able  to  produce  at  least  15%  more  completed  missions  over  any  given 
time  period  than  a  conventional  system. 

A  fully  ISHM  enabled  design  will  be  able  to  reduce  the  system  attrition  rate  by  over  10%  compared  to  a 
conventional  system. 

Fully  integrated  health  management  can  lead  to  dramatic  reductions  in  weight,  cost,  design  and 
development  flow  time,  certification  and  qualification  time  and  cost.  Not  to  mention  changing  the 
fundamental  paradigm  driving  the  way  systems  are  designed. 
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2.6.3  The  ISHM  Technology  Development  Roadmap 

The  research  conducted  by  MacConnell  also  proposed  follow-on  ISHM  capability 
planning  in  the  form  of  an  ISHM  Capabilities  Roadmap.  Refer  to  Figure  10.  This  serves 
as  a  foundation  for  which  ISHM  related  research  efforts  can  be  focused  and  provided  an 
impetus  for  this  thesis  effort.  The  architecture  developed  through  this  research  aimed  to 
validate  the  business  case  of  ISHM  through  improved  mission  availability  (with  CBM) 
and  improved  mission  success  (with  improved  safety  of  UAS).  In  the  longer  tenn,  the 
FMECA  architecture  proposed  could  also  be  utilized  in  UAS  design  paradigms. 

This  will  be  consistent  with  the  objective  of  this  research  in  deriving  performance 
requirements  for  components  of  an  ISHM-enabled  UAS  for  airworthiness  certification. 
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Figure  10  -  ISHM  Capabilities  Roadmap  [MacConnell,  2006] 
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2.7  Successful  Practice  of  Simulation  Experiments 

Simulation  is  a  powerful  tool  for  evaluation  and  analysis  of  new  system  architecture  and 
designs,  modifications  to  existing  designs  and  proposed  changes  to  operational  rules. 
Conducting  a  valid  simulation  experiment  is  both  an  art  and  a  science.  As  a  systems 
engineer,  it  is  important  to  recognize  that  the  success  of  simulations  involved  much  more 
than  the  technical  aspects  that  one  has  been  trained  in.  This  section  presents  the  common 
pitfalls  in  perfonning  simulation  studies  and  identifies  approaches  for  avoiding  them 
[Sadowski  and  Grabau,  2004]. 

2.7.1  What  is  Success? 

First  and  foremost,  a  successful  simulation  project  is  one  that  delivers  useful  information 
at  the  appropriate  time  to  support  a  meaningful  decision. 

The  Right  Information  -  presenting  infonnation  from  the  perspective  of  the  decision 
makers,  in  the  proper  context  of  what  they  will  be  doing  with  this  information  to  deliver 
value  to  the  proposition. 

The  Right  Timing  -  intuitively,  a  high-fidelity  answer  that  is  too  late  to  influence  a 
decision  is  not  nearly  as  good  as  a  rough  estimate  that  is  in  time  to  help.  In  addition, 
preliminary  insights  always  serve  their  purpose  in  a  project  for  decision  makers  to  steer 
their  focus. 
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The  Right  Decision  -  this  aspect  may  not  be  within  your  span  of  control,  but  on-time 
information  needs  to  be  delivered  to  the  right  person  in  the  right  context  for  the  right 
decision  to  be  made. 


2.7.2  Pitfalls 

Table  5  summarizes  the  potential  pitfalls  faced  in  simulation  projects  as  presented  by 
Sadowski  and  Garbau. 


Table  5  -  Common  Pitfalls  of  Simulation  Projects  |Sadowski  and  Garbau,  2004] 


Tackling  the  Wrong  Problem 

Step  back  and  double  check  that  simulation  is  the  best  tool  for  the  problem 

Ill-defined  scope  that  might  be  too  ambitious 

Working  on  the  Right  Problem  at  the  Wrong  Time 

Designers  are  still  considering  widely  differing  ideas 

Fundamental  systemic  problems  not  resolved 

Late  request  -  panic  call  for  information 

Missing  the  Warning  Signs  of  ‘Data  Woes’ 

More  often,  too  little  data  is  available  for  simulation  -  or  getting  the  required  data  might  be  time 
consuming.  Important  to  establish  data  needs  early 

Too  much  data  -  identifying  valid  and  accurate  data  from  multiple  sources 

Letting  the  Window  of  Opportunity  Close 

Getting  lost  in  detail  -  adding  too  much  details  into  the  simulation  just  because  you  could 

Leaving  analysis  till  the  end  -  unable  to  draw  valuable  conclusions  from  simulation  results 

Having  too  much  fun  with  animation  -  distracted  with  the  software  tool  itself 

Testing  only  at  the  end  of  a  project  -  validation  and  verification  should  be  a  continuous  process 
throughout  the  project 
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2.7.3  Approach  to  Avoid  Pitfalls 

The  final  part  of  this  section  lists  a  few  simple  habits  (applicable  in  all  projects)  that  help 
to  circumvent  the  above-mentioned  pitfalls  [Sadowski  and  Grabau,  2004].  These  may 
seem  intuitive  to  all  readers  but  internalizing  these  habits  amidst  external  demands, 
pressures  and  distractions  may  be  challenging. 

•  Establish  a  clear  focus; 

•  Plan  carefully  and  thoroughly; 

•  Build  a  realistic  timeline; 

•  Constantly  review  and  reassess. 

This  research  will  employ  a  discrete-event  simulation  to  implement  the  analytic 
architecture.  Being  aware  of  the  potential  pitfalls  of  a  simulation  project  and  the  various 
approaches  to  mitigate  them  seme  as  constant  beacons  in  ensuring  a  positive  outcome 
for  this  research. 

2.8  Literature  Review  Summary 

This  section  presented  the  existing  state  of  ISHM  implementation  through  its  various 

definitions  and  applications.  Following  which,  a  typical  ISHM  architecture  was  presented 

with  the  following  elements:  a  sensor  suite  that  detects/identifies  critical  system 

conditions,  a  management  component  that  included  sensor  data  processing,  diagnostic 

and  prognostic  algorithms  to  identify  current  or  incipient  faults,  and  a  reasoner  to  select 

the  appropriate  mitigation  steps  to  execute.  Key  airworthiness  and  system  safety  concepts 

aligned  to  the  Air  Force’s  safety  objectives  were  also  presented;  with  specific  focus  on 

34 


UAS  safety  in  tenns  of  a  ground  impact  analysis.  The  FMECA  methodology  was  also 
introduced  and  a  preliminary  hazard  assessment  conducted  by  NASA  on  a  generic  UAS 
was  also  presented.  The  review  proceeded  to  recognize  the  benefits  of  an  ideal  ISHM 
system  and  highlighted  the  business  case  and  ISHM  capabilities  roadmap  to  substantiate 
follow-on  ISHM  research  efforts.  Finally,  the  last  section  of  the  literature  review 
discussed  the  potential  pitfalls  and  mitigating  approach  of  a  simulation  experiment,  which 
would  be  the  methodology  employed  to  validate  the  proposed  architecture  of  this  thesis. 
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III.  Architecture  Definition 


3.1  Overview 

This  chapter  describes  the  analytic  architecture  that  provides  the  basis  of  the  discrete 
event  simulation  model.  The  architecture  is  developed  in  accordance  with  the  Department 
of  Defense  Architecture  Framework  (DoDAF).  The  chapter  provides  a  preview  through 
the  concept  of  operations  for  the  analytic  architecture.  Thereafter,  specific  architectural 
products  that  are  of  relevance  to  the  simulation  model  are  created.  Finally,  the 
architecture  is  translated  to  logical  flows  pertaining  to  the  implementation  of  the 
simulation  model. 

3.2  Concept  of  Operations 
3.2.1  Issue 

A.  Problem  Statement 

As  presented  in  the  literature  review,  health  management  at  present  is  still  viewed 
primarily  as  a  means  for  detecting  fault  and/or  failures  in  support  of  maintenance 
activities;  with  health  management  systems  being  an  ‘after-thought’  of  system  designs. 
This  led  to  the  development  of  health  management  systems  focused  on  individual 
components  of  subsystems  with  data  generated  being  typically  viewed  only  within  the 
context  of  that  subsystem.  An  ISHM  architecture  for  analysis  of  UAS  failure  modes  will 
provide  a  means  for  initial  airworthiness  certification  of  ISHM-enabled  UAS  designs  - 
by  determining  perfonnance  requirements  of  subsystems  or  components. 
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B.  Purpose  of  the  CONOPS 

The  purpose  of  this  CONOPS  is  to  articulate  how  the  architecture  will  enhance 
airworthiness  (and  other  secondary  benefits)  through  the  analysis  of  failure  inodes  of  a 
generic  UAV.  Specifically,  these  are  the  research  questions  that  the  architecture  aims  to 
answer: 

(1)  What  are  the  perfonnance  characteristics  of  ISHM  to  ensure  continued 
airworthiness  of  the  UAS? 

(2)  How  will  ISHM  provide  a  business  case  to  improve  the  level  of  UAS  self¬ 
autonomy? 

(3)  What  are  the  potential  impacts  of  ISHM  to  maintenance  practices  and  life 
cycle  costs? 

3.2.2  Overview 

A.  Synopsis 

The  proposed  architecture  shall  optimize  existing  UAS  designs  through  the  incorporation 
of  a  typical  ISHM  configuration  (i.e.  sensors  suite,  diagnostic/prognostic  algorithms  and 
a  decision  reasoner).  With  available  FMECA  data  in  terms  of  possible  failure  conditions 
as  input,  the  architecture  will  then  evaluate  the  effectiveness  of  ISHM  in  terms  of  its 
resultant  reliability  over  the  lifetime  of  the  UAS.  The  architecture  will  also  consider  the 
degradation  of  the  ISHM  suite  in  terms  of  missed  detection  and  false  alarm  rates  and  their 
effects  on  maintenance  policies.  The  implementation  of  the  analytic  architecture  for 
ISHM-enabled  UAS  shall  provide  basis  for: 
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(i)  initial  airworthiness  certification  during  UAS  induction, 

(ii)  continued  airworthiness  certification  when  field  reliability  data  are  available, 
and 

(iii)  change  in  maintenance  policies  and  potential  benefits  in  terms  of  increased 
availability  and  cost  savings. 

B.  Description  of  Military  Challenge 

Besides  quantitative  benefits  in  terms  of  system  availability,  mission  success  and  related 
cost  savings,  ISHM  presents  mid-  and  long-term  benefits  in  the  design  paradigm  through 
expansion  of  operational  envelopes,  reduced  safety  factors  and  revolutionary  certification 
techniques.  In  seeking  to  improve  the  trust  in  autonomy  of  UAS,  this  analytic  architecture 
shall  examine  the  target  reliability  of  an  ISHM-enabled  UAS  over  its  lifetime. 

C.  Desired  Effects 

The  architecture  shall  provide  a  baseline  analytical  model  for  a  generic  UAS.  Various 
failure  conditions,  their  failure  probability  distributions  and  assigned  criticalities  provide 
inputs  for  the  analysis.  ISHM  effectiveness  can  be  modeled  through  the  strength  of  its 
diagnostics  and  prognostics  algorithms,  i.e.  probability  of  an  accurate  ISHM  deduction. 
On  the  other  hand,  ISHM  degradation  is  compared  against  established  thresholds  of  false 
alarms  and  missed  detections.  Output  and  sensitivity  analysis  on  measures  of 
perfonnance,  such  as  UAS  life  expectancy  and  cost  of  maintenance  over  its  lifetime,  can 
then  be  performed  to  detennine  ideal  perfonnance  requirements  and  maintenance  policies 
for  specific  UAS  designs. 
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3.2.3  Context 


A.  Time  Horizon 

The  architecture  and  preliminary  analysis  should  be  completed  by  end  July  2013  with  the 
research  out-brief  scheduled  in  end  Aug  2013.  The  ISHM  technology  relevant  to  the 
implementation  of  this  analytic  model  should  be  available  in  the  next  5  to  15  years. 


B.  Assumptions 


(i)  Without  actual  FMECA  data,  it  will  not  be  possible  to  model  and/or  evaluate 
all  the  failure  modes  for  a  typical  UAS.  As  such,  only  selected  critical  sub¬ 
system  failures  affecting  airworthiness  of  the  UAS  shall  be  evaluated  using 
the  model. 

(ii)  Without  actual  sensor  performance  data,  theoretical/nominal  thresholds  will 
need  to  be  assumed  to  model  sensor  degradation  in  terms  of  False  Alann  Rate 
(FAR)  and  missed  detections. 


C.  Risks 

Absence  of  real-world  UAS  design  data  may  raise  uncertainties  on  the  accuracy  of  the 
output  metrics.  However,  this  research  seeks  to  establish  a  sound  analytical  architecture 
that  would  serve  its  intended  benefits  through  appropriate  sensitivity  analyses. 


3.2.4  Employment  Concept 

A.  Critical  Capabilities 

The  critical  capabilities  needed  to  meet  the  desired  end  state  of  this  architecture  include: 


(i)  Flexibility  -  able  to  be  further  customized  for  specific  UAS  configurations 
with  associated  FMECA. 
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(ii)  Analysis  Support  -  regardless  of  evaluation/simulation  tool,  the  architecture 
will  be  able  to  support  the  associated  software  analysis  and  evaluation  to  yield 
useful  results. 

B.  Enabling  Capabilities 

In  order  to  better  implement  the  architecture,  a  formal  Failure  Mode,  Effect,  and 
Criticality  Analysis  (FMECA)  must  be  performed  on  the  UAS.  This  is  an  iterative 
process  throughout  the  System  Life  Cycle  that  identifies  failure  modes,  assesses  their 
probabilities  of  occurrence,  criticalities  and  their  effects  on  the  system.  The  results  of  the 
FMECA  should  identify  critical  sub-systems  or  components  that  ISHM  needs  to  monitor 
and/or  control;  and  guide  the  diagnostic  and  prognostic  algorithms  required  for  effective 
health  management. 

C.  End  State 

A  system-wide  analytic  architecture  for  an  ISHM-enabled  generic  UAS  -  capable  of 
delivering  design  decisions  based  on  failure  modes  of  UAS,  which  in  turn  ensures 
continued  airworthiness  of  UAS  operations. 

3.3  Architecture  Views 

3.3.1  Operational  Activity  Decomposition  Tree  (OV-5a) 

Figure  11:  Describes  the  operational  activities  organized  in  a  hierarchical  structure. 
There  are  two  top-level  operational  activities  for  the  architecture;  namely  “Perform 
Generic  UAS  Activities”  and  “Perform  ISHM”.  The  fonner  is  applicable  to  all  UAS  and 
decomposes  into  lower-level  activities  such  as  “Perform  Top-Level  UAS  In-Flight 
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Functions”  -  Aviate,  Navigate,  Communicate  and  Mitigate  [Hayhurst  et  al.,  2007], 
“Perform  Failure  Detection”  and  “Perform  Failure  Assessment”.  In  addition,  a  generic 
UAS  upon  confirmation  of  failure,  will  “Perform  Subsystem  Maintenance”  upon  landing. 
The  analytic  architecture  will  also  need  to  “Record  UAS  Operational  Lifetime”  to 
detennine  scheduled  maintenance  requirements  and  more  importantly,  the  accumulated 
life  expectancy  of  the  UAS. 

The  top-level  operational  activity  -  “Perform  ISHM”  is  only  applicable  for  ISHM- 
enabled  UAS.  It  decomposes  into  two  lower-level  activities  -  “Perform  Diagnostics  and 
Prognostics  Assessment”  determines  the  accuracy  and  effectiveness  of  ISHM  algorithms, 
while  “Perform  ISHM  Maintenance”  repairs/replaces  ISHM  components  when  false 
alann  and  missed  detection  thresholds  are  exceeded. 


41 


Figure  11  -  OV-5a:  Operational  Activity  Decomposition  Tree 


OV-5  Operational  Activity  Model  / 


AO  Evaluate  Airworthiness 
Requirements  of 
ISHM-Enabled  UAS 

- 5 - 


A1  Perform  Generic 
UAS  Activities 
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Failure 
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A13.1  Assess 
Failure  Probability 


A13.2  Assess 
Failure  Severity 


A13.3  Assign 
Failure  Latency 


A13.4  Assess 
Latency  Effect 


A1.4  Perform 
Subsystem 
Maintenance 


A14.1  Perform 
Scheduled 
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ISHM) 


A14.2  Replace  Failed 
Component 


A14.3  Replace  Faulty 
Sensor 
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'  ISHM  False  Alarm 
v _ Rate 


A2.2  Perform 
ISHM 

Maintenance 
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Maintenance 


A22.2  Record  ISHM 
Maintenance  Time 


^  A22.3  Record  ISHM  ^ 
-  Maintenance  Activities 
&  Cost 

\ _ _ / 


A14.6  Record  Sensor 
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V _ / 


A14.7  Record  CND 
Count  and 
Troubleshooting 
Downtime 


^  A14.8  Record  UAS  ^ 
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3.3.2  Operational  Rules  Model  (OV-6a) 


Table  6:  Describes  the  business  rules  that  constrain  operations  for  an  ISHM-enabled 
UAS. 

Based  on  the  results  of  the  various  conditions  listed  in  the  “Condition  Stub”,  each  of  the 
19  operational  rules  detennines  a  set  of  specific  actions  in  the  “Action  Stub”  to  be 
executed  by  the  simulation. 

Table  6  -  OV-6a:  Operational  Rules  Model  (ISHM-Enabled  UAS) 


Rules  and  Decision  Analysis  Table  (ISHM-Enabled  UAS) 

Condition  Stub 

Condition  Matrix 

R1  |  R2  |  R3  |  R4  |  R5  |  R6  |  R7  |  R8  |  R9  |  RIO  |  Rll  |  R12  |  R13  |  R14  |  R15  |  R16  |  R17  |  R18  |  R19 

Determine  if  Failure  Condition  is  Present  During  Flight 

Failure  Occurred?  |y|y|y|y|y|y|y|y|y|y|n|n|n|n|n|n|n|n|n 

Determine  if  UAS  is  Aware  of  Present  Failure  Condition 

Is  Failure  Detected  by  Isolated  Subsystem  Sensor? 

Y 

Y 

Y 

Y 

Y 

N 

N 

N 

N 

N 

Y 

Y 

Y 

Y 

Y 

Y 

N 

N 

N 

Is  Failure  Ascertained  by  ISHM? 

Y 

N 

N 

N 

N 

Y 

Y 

N 

N 

N 

Y 

Y 

Y 

Y 

N 

N 

Y 

Y 

N 

Assess  Failure  Severity  and  Latency  Effect 

Is  Failure  Catastrophic? 

Y 

N 

N 

Y 

N 

Is  Failure  Latent  (occurred  in  previous  flights  but  not 
detected/ascertained)? 

Y 

N 

N 

Y 

N 

Determine  Maintenance  Requirements  of  Existing  Sensors 

Is  Sensor  Miss  Rate  Exceeded? 

N 

N 

N 

Y 

N 

N 

Is  Sensor  False  Alarm  Rate  Exceeded? 

N 

Y 

Y 

N 

N 

Y 

N 

N 

N 

N 

Determine  Maintenance  Requirements  of  ISHM 

Is  ISHM  Miss  Rate  Exceeded? 

N 

Y 

N 

N 

N 

N 

Is  ISHM  False  Alarm  Rate  Exceeded? 

N 

Y 

N 

Y 

N 

N 

N 

Y 

N 

N 

Action  Stub 

Action  Matrix 

Perform  Failure  Assessment 

A13.2  Assess  Failure  Severity 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

A13.3  Assign  Failure  Latency 

X 

A13.4  Assess  Latency  Effect 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Decide  UAS  Recovery  State 

UAS  Recovering  Code  1 

[All.  1  Aviate  -  No  Discrepancy  Reported] 

X 

X 

UAS  Recovering  Code  2 

[A  11.1  Aviate  -  Minor  Discrepancy  Reported  But  Does  Not 
Affect  Current  Mission] 

X 

X 

X 

X 

UAS  Recovering  Code  3 

[A11.4  Mitigate  -  Discrepancy  Reported  that  Require 
Rectification  Before  Next  Mission] 

X 

X 

X 

X 

X 

X 

X 

X 

X 

UAS  Mishap 

[A15.2  Record  UAS  Accumulated  Life  -  Terminated] 

X 

X 

X 

X 

Degradation  of  Isolated  Subsystem  Sensors 

A12.2  Determine  Sensor  Missed  Detection  Rate 

[After  increasing  count] 

X 

X 

X 

X 

A12.3  Determine  Sensor  False  Alarm  Rate 
[after  increasing  count] 

X 

X 

X 

X 

X 

X 

Degradation  of  ISHM  Suite 

A21.2  Determine  ISHM  Missed  Detection  Rate 
[after  increasing  count] 

X 

X 

X 

X 

X 

X 

A21.3  Determine  ISHM  False  Alarm  Rate 
[after  increasing  count] 

X 

X 

X 

X 

X 

X 

Perform  Required  Maintenance 

A14.4Troubeshootfor  Cannot  Duplicate  (CND) 

X 

X 

X 

X 

X 

X 

X 

X 

A14.7  Record  CND  Count  and  Troubleshooting  Downtime 

X 

X 

X 

X 

X 

X 

A14.2  Replace  Failed  Component 

X 

X 

X 

X 

X 

A14.3  Replace  Faulty  Sensor 

X 

X 

X 

X 

A22.1  Perform  Unscheduled  ISHM  Maintenance 

X 

X 

X 

X 

Record  UAS  Operational  and  Maintenance  Life 

A15.1  Determine  Flight  Time  Based  on  Landing  Codes 
[Normal  Flight  Time  for  Codes  land  2] 

X 

X 

X 

X 

X 

X 

A15.1  Determine  Flight  Time  Based  on  Landing  Codes 
[Reduced  Flight  Time  for  Code  3  and  Mishap] 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Notes: 


1)  in  Condition  Matrix  indicates  condition  does  not  matter  for  recommended  set  of  actions. 

2)  'X'  in  Action  Matrix  indicates  specified  actions  to  be  carried  out  or  transition  into  pre-defined  states  based  on  conditions. 
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Table  7:  Describes  the  business  rules  that  constrain  operations  for  a  non-ISHM  UAS. 
Based  on  the  results  of  the  various  conditions  listed  in  the  “Condition  Stub”,  each  of  the 


12  operational  rules  detennines  a  set  of  specific  actions  in  the  “Action  Stub”  to  be 
executed  by  the  simulation.  With  a  non-ISHM  UAS,  fewer  conditions  and  actions  exist 
due  to  absence  of  an  ISHM  failure  (or  non-failure)  declaration  and  the  associated  ISHM 
maintenance  actions,  therefore  there  will  be  fewer  rules  than  an  ISHM-enabled  UAS. 


Table  7  -  OV-6a:  Operational  Rules  Model  (Non-ISHM  UAS) 


Rules  and  Decision  Analysis  Table  (Non-ISHM  UAS) 

Condition  Stub 

Condition  Matrix 

R1  |  R2  |  R3  |  R4  |  R5  |  R6  |  R7  |  R8  |  R9  |  RIO  |  Rll  |  R12 

Determine  if  Failure  Condition  is  Present  During  Flight 

Failure  Occurred? 

y|y|y|y|y|y|n|n|n|n|n|n 

Determine  if  UAS  is  Aware  of  Present  Failure  Condition 

Is  Failure  Detected  by  Isolated  Subsystem  Sensor? 

y|y|n|n|n|n|y|y|y|y|n|n 

Assess  Failure  Severity  and  Latency  Effect 


Is  Failure  Catastrophic? _ Y  -  N  _ N_ 

Is  Failure  Latent  (occurred  in  previous  flights  but  not  Y  N  N 

detected/ascertained)? 


Determine  Maintenance  Requirements  of  Existing  Sensors 


Is  Sensor  False  Alarm  Rate  Exceeded?  |-|-|-|-|-|-|y|y|n|n|n|n 

Determine  Scheduled  Maintenance  Requirements  of  UAS 

Is  UAS  Due  for  Scheduled  Maintenance? 

y|n|-|-|y|n|y|n|y|n|y|n 

Action  Stub 

Action  Matrix 

Perform  Failure  Assessment 

A13.2  Assess  Failure  Severity 

X 

X 

X 

X 

X 

X 

A13.3  Assign  Failure  Latency 

X 

A13.4  Assess  Latency  Effect 

X 

X 

X 

X 

X 

X 

Decide  UAS  Recovery  State 

UAS  Recovering  Code  1 

[All.l  Aviate  -  No  Discrepancy  Reported] 

X 

X 

X 

X 

UAS  Recovering  Code  3 

[A11.4  Mitigate  -  Discrepancy  Reported  that  Require 
Rectification  Before  Next  Mission] 

X 

X 

X 

X 

X 

X 

UAS  Mishap 

[A15.2  Record  UAS  Accumulated  Life  -  Terminated] 

X 

X 

Degradation  of  Isolated  Subsystem  Sensors 

A12.3  Determine  Sensor  False  Alarm  Rate 
[after  increasing  count] 

X 

X 

X 

X 

Perform  Required  Maintenance 

A14.4Troubeshoot  for  Cannot  Duplicate  (CND) 

X 

X 

X 

X 

A14.7  Record  CND  Count  and  Troubleshooting  Downtime 

X 

X 

X 

X 

A14.1  Perform  Scheduled  Maintenance  (Without  ISHM) 
[Reset  Maintenance  Schedule] 

X 

X 

X 

X 

X 

A14.2  Replace  Failed  Component 

X 

X 

A14.3  Replace  Faulty  Sensor 

X 

X 

Record  UAS  Operational  and  Maintenance  Life 

A15.1  Determine  Flight  Time  Based  on  Landing  Codes 
[Normal  Flight  Time  for  Codes  1  and  2] 

X 

X 

X 

X 

A15.1  Determine  Flight  Time  Based  on  Landing  Codes 
[Reduced  Flight  Time  for  Code  3  and  Mishap] 

X 

X 

X 

X 

X 

X 

X 

X 

A15.3  Determine  Time  for  Scheduled  Maintenance 
[Add  Flight  Time  to  Maintenance  Schedule] 

X 

X 

X 

X 

Notes: 


1)  in  Condition  Matrix  indicates  condition  does  not  matter  for  recommended  set  of  actions. 

2)  'X'  in  Action  Matrix  indicates  specified  actions  to  be  carried  out  or  transition  into  pre-defined  states  based  on  conditions. 
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3.3.3  State  Transition  Description  (OV-6b) 

Figure  12:  Describes  the  business  process  (activity)  responses  to  presence  of  failure 
condition. 

This  state  transition  diagram  provides  a  temporal  view  of  the  analytic  architecture  in  the 
event  of  a  failure  condition.  The  left-hand  side  (LHS)  of  the  diagram  depicts  the  state 
transition  of  an  ISHM-enabled  UAS  while  the  right-hand  side  (RHS)  shows  the 
corresponding  state  transition  of  a  non-ISHM  UAS.  Consistent  with  the  operational  rules 
presented  in  the  OV-6a,  the  architecture  transits  through  the  various  states  based  on  the 
presented  conditions.  These  states  include  the  UAS  recovering  with  various  landing 
codes  based  on  failure  detection  by  the  subsystem  sensors  and  failure  detennination  by 
ISHM  (LHS).  In  addition,  based  on  the  landing  codes,  appropriate  maintenance  actions 
on  the  subsystem  and/or  ISHM  are  executed.  Specifically  for  a  non-ISHM  UAS,  there  is 
provision  for  scheduled  maintenance.  In  the  event  that  the  failure  condition  is  not 
detected  (by  both  the  subsystem  sensor  and  ISHM),  the  model  transits  into  a  state  to 
determine  the  latency  effect  of  the  missed  failure  condition.  If  a  missed  failure  condition 
is  catastrophic  or  latent  (to  the  extent  that  it  becomes  critical),  the  model  tenninates  with 
a  UAS  mishap. 

Figure  13:  Describes  the  business  process  (activity)  responses  to  absence  of  failure 
condition. 

This  state  transition  diagram  provides  a  temporal  view  of  the  analytic  architecture  in  the 

absence  of  a  failure  condition.  Without  actual  failures,  this  model  does  not  result  in  a 

UAS  mishap.  However,  this  view  will  be  more  concerned  with  false  alanns  and  Cannot- 
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Duplicate  (CND)  that  generate  unnecessary  maintenance  activities.  Otherwise,  the  states 
presented  in  both  state  transition  diagrams  are  similar. 


OV-6b  State  Transition  Description  / 


Figure  12  -  OV-6b:  State  Transition  Description  (Failure  Condition  Present) 
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OV-6b  State  Transition  Description  ^ 


Figure  13  -  OV-6b:  State  Transition  Description  (Failure  Condition  Absent) 
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3.4  Discrete  Event  Simulation  Model 


3.4.1  Logic  Flow 

Based  on  the  analytic  architecture,  flowcharts  are  drawn  up  to  present  the  logical 
sequence  that  guide  the  development  of  the  simulation  model.  Four  logical  sequences  are 
presented  in  the  following  figures. 

•  Failure  Condition  Present  for  an  ISHM-Enabled  UAS:  Refer  to  Figure  14. 

•  Failure  Condition  Present  for  a  Non-ISHM  UAS:  Refer  to  Figure  15. 

•  Failure  Condition  Absent  for  an  ISHM-Enabled  UAS:  Refer  to  Figure  16. 

•  Failure  Condition  Absent  for  a  Non-ISHM  UAS:  Refer  to  Figure  1 7. 

3.4.2  Model  Parameters 

Input  modeling  and  output  analysis  are  two  important  phases  of  a  simulation  project. 
Based  on  the  model  logic,  the  required  input  parameters  and  desired  output  performance 
measures  are  defined  in  Table  8. 

3.4.3  Sensitivity  Analysis  -  Design  of  Experiments  (DOE) 

The  benefits  of  this  research  effort  will  be  based  on  the  analysis  of  the  simulation  results. 
Representative  data  for  a  UAS  will  be  fed  into  the  model  and  DOE  techniques  will  be 
employed  to  detennine  situations  where  the  life  expectancy  of  an  ISHM-enabled  UAS 
can  be  maximized  based  on  the  various  UAS  design  factors.  The  secondary  objective  of 
the  analysis  will  be  to  establish  a  business  case  for  ISHM  by  minimizing  the  cost  of  a 
condition-based  maintenance  program  with  onboard  ISHM. 


48 


Failure 

Occurred? 


-►  See  Figure  1 6 


Determined 

bylSHM? 


Yes 

I 

Determine  Failure 
Severity  Category  & 
Latency  Effect 

i 

s'  Failure 
Detected? 


Isolated  Subsystem 
Sensor 

No 


ISHM  Diagnostics 
and/or  Prognostics 


Determined 

bylSHM? 


Latent  Failure? 


Latent  Failure? 


No 

Y 


No 

Y 


UAS  Recovered 
Code  3 


Add  ‘Reduced’ 
Flight  Time  to  UAS 
Life 


No 

▼ 


Is  Failure 
Catastrophic?, 


UAS  Recovered 
Code  2 


Add  Normal 
Flight  Time  to  UAS 
Life 


Terminate 
UAS  Life 


Is  Failure 
Catastrophic? 


No 

▼ 


UAS  Recovered 
Code  1 


Add  Normal 
Flight  Time  to  UAS 
Life 


UAS  Recovered 
Code  3 


Add  ‘Reduced’ 
Flight  Time  to  UAS 
Life 


t 

Add 

Unscheduled 

Component 

Replacement 

Activity 


t 

Add 

Unscheduled 

Component 

Replacement 

Activity 


Assign 

Latent  Failure 


Y 

Add 

Unscheduled 

Component 

Replacement 

Activity 


_ I _ 

Add 

UAS  Downtime 


_ 1 _ 

Add 

UAS  Downtime 

▼ 

Add 

ISHM  Miss 


ISHM 
Miss  Rate 
^Exceeded'?/ 


Add 

Add 

Unscheduled 

Unscheduled 

ISHM 

Sensor 

Maintenance 

Maintenance 

Activity 

Activity 

_ I _ 

, 

Reset  ISHM 

Reset  Sensor 

Miss  Rate 

Miss  Rate 

▼ 

Add 

UAS  Downtime 


Add 

UAS  Downtime 


_ I _ 

Add 

UAS  Downtime 

▼ 

Add 

Sensor  Miss 


Sensor 
Miss  Rate 
Exceeded? 


_ * _ 

Next  Flight 


Figure  14  -  Simulation  Model  Flowchart:  Failure  Condition  Present  for  an  ISFiM-Enabled  UAS 
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Figure  15  -  Simulation  Model  Flowchart:  Failure  Condition  Present  for  a  Non-ISHM  UAS 
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Figure  16  -  Simulation  Model  Flowchart:  Failure  Condition  Absent  for  an  ISHM-Enabled  UAS 
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Figure  17  -  Simulation  Model  Flowchart:  Failure  Condition  Absent  for  a  Non-ISHM  UAS 
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Table  8  -  Simulation  Model  Parameters 


UAS  Properties 
(Input  Parameters) 

Definition 

P(Failure) 

The  probability  of  a  failure  occurring  for  a  given  sortie. 

P(Failure  is  Catastrophic) 

Given  that  a  failure  occurs,  the  probability  of  it  being  catastrophic,  i.e.  resulting 
in  mishap  if  undetected. 

Latency  Threshold 

The  accumulated  latency  effect  that  determines  criticality  of  a  failure  condition 
every  time  it  is  not  detected  in  previous  flights.  A  latent  failure  that  reaches  its 
pre-determined  Latency  Threshold  due  to  prior  missed  detections  will  be 
upgraded  to  a  catastrophic  failure  resulting  in  a  mishap  -  such  as  in  the  case  of 
a  crack  propagating  beyond  its  critical  crack  length. 

P(  Sensor  Detection) 

The  probability  of  the  sensor  detecting  a  failure  given  that  a  failure  condition 
exists. 

P(Sensor  False  Alarm) 

The  probability  of  the  sensor  detecting  a  failure  given  that  there  is  no  failure 
condition. 

Sensor  Missed  Detection 
Threshold 

The  performance  specification  of  the  sensor  in  terms  of  missed  detections; 
exceeding  which  sensor  maintenance  is  to  be  carried  out. 

Sensor  False  Alarm 
Threshold 

The  performance  specification  of  the  sensor  in  terms  of  false  alarms;  exceeding 
which  sensor  maintenance  is  to  be  carried  out. 

Scheduled  Maintenance 
Interval 

For  a  non-ISHM  UAS,  the  interval  between  scheduled  maintenance  (in  hours). 

Sortie  Flight  Time 
(Normal) 

The  duration  of  flight  (in  hours)  for  a  UAS  mission  that  recovered  Code  1  or 
Code  2. 

Sortie  Flight  Time 
(Reduced) 

The  truncated  duration  of  flight  (in  hours)  for  a  UAS  mission  that  recovered 
Code  3. 

ISHM  Properties 
(Input  Parameters) 

Definition 

P(ISHM  Confidence) 

The  probability  of  ISHM  declaring  that  a  failure  condition  has  occurred  when 
an  actual  failure  condition  exists  -  an  indication  of  the  strength  of  the 
diagnostic  and  prognostic  algorithms. 

P(ISHM  False  Alarm) 

The  probability  of  ISHM  declaring  that  a  failure  condition  has  occurred  when 
there  is  no  actual  failure. 

ISHM  Missed  Detection 
Threshold 

The  performance  specification  of  ISHM  in  terms  of  missed  detections; 
exceeding  which  ISHM  maintenance  is  to  be  carried  out. 

ISHM  False  Alarm 
Threshold 

The  performance  specification  of  ISHM  in  terms  of  false  alarms;  exceeding 
which  ISHM  maintenance  is  to  be  carried  out. 

Expected  Model 
Output  Measures 

Definition 

Expected  System 
Lifetime 

The  expected  lifetime  of  the  UAS  (in  hours)  based  on  accumulation  of 
airworthy  sorties. 

Number  of  CND 

The  expected  number  of  Cannot-Duplicate  (CND)  cases  that  contribute  to 
unnecessary  maintenance. 
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Scheduled  Maintenance 
Actions 

The  expected  number  of  scheduled  maintenance  actions  for  a  non-ISHM  UAS. 

Unscheduled 
Maintenance  Actions 

The  expected  number  of  component  replacements  due  to  actual  failures. 

Sensor  Maintenance 
Actions 

The  expected  number  of  sensor  maintenance  actions  due  to  exceedance  of 
missed  detection  and  false  alarm  thresholds. 

ISHM  Maintenance 
Actions 

The  expected  number  of  ISHM  maintenance  actions  due  to  exceedance  of 
missed  detection  and  false  alarm  thresholds. 

UAS  Downtime 

This  is  a  calculated  parameter  based  on  fixed  maintenance  times  for  the  above 
categories  of  maintenance  actions. 

Model  Properties 

Definition 

Number  of  Iterations 

The  number  of  iterations  to  execute  the  simulation  model  to  achieve  greater 
statistical  accuracy. 

3.5  Concordance  and  Consistency 

The  fundamental  principle  in  architecting  lies  in  ensuring  concordance  between 
architectural  products  -  so  that  end-users  (customers,  builders,  architects)  looking  at 
different  products  will  have  a  common  picture  of  the  desired  architecture.  In  this  research 
effort,  where  the  analytic  architecture  is  applied  to  a  simulation  model,  greater  care  must 
be  taken  to  ensure  consistency  between  the  architecture  and  the  translated  logic  flow  that 
is  to  be  implemented  within  the  simulation  model.  In  order  to  demonstrate  the  adherence 
of  concordance  and  consistency  principles  in  the  research  methodology,  this  section 
identified  a  specific  logic  path  from  Figure  14,  and  traced  its  relevance  and  consistency 
with  the  three  architectural  products  presented  (refer  to  Figure  18): 

•  OV-5a  (Operational  Activity  Decomposition  Tree)  -  The  actions  executed  within 
the  logic  path  are  consistent  with  defined  operational  activities; 

•  OV-6a  (Operational  Rules  Model)  -  The  actions  executed  based  on  decisions 
within  the  logic  path  are  consistent  with  the  defined  operational  rule  for  the 
identified  path; 
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•  0V-6b  (State  Transition  Description)  -  The  logical  sequence  presented  by  the 

logic  path  is  consistent  with  the  defined  states,  conditions  and  associated 
operational  activities. 
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Figure  18  -  Demonstrating  Concordance  and  Consistency 
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3.6  Architecture  Summary 

This  chapter  defined  the  architecture  for  this  research  originating  from  the  CONOPS  that 


drove  the  relevant  analytic  architectural  products,  and  subsequently  translated  into  logical 
sequences  for  the  implementation  of  the  simulation  model.  While  various  heuristics 
guided  the  art  of  architecture,  the  principle  of  concordance  between  products  was 
enforced  throughout  the  architecting  process  -  ensuring  consistent  representation  in  the 
development  of  the  simulation  model.  Architecting  for  the  purpose  of  sound  analysis 
provided  a  well-grounded  methodology  for  this  research. 
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IV.  Analysis  and  Results 


4.1  Chapter  Overview 

This  chapter  will  present  the  implementation  of  the  analytic  architecture  using  Arena  ® 
Version  14.5  and  the  analysis  of  the  discrete  event  simulation  model  using 
representative  UAS  data  for  both  ISHM-enabled  and  non-ISHM  UAS  models. 

4.2  Input  Modeling 

Input  analysis  is  the  process  of  characterizing  input  variables  -  identifying  their  possible 
values  and  associated  probabilities.  Input  models  provide  the  driving  force  for  a 
simulation,  and  identifying  the  right  input  variables  will  form  the  basis  of  sensitivity 
analysis  at  the  end  of  a  simulation  project.  Choosing  the  appropriate  distributions  for 
input  data  is  a  major  task,  as  faulty  models  will  lead  to  outputs  whose  interpretation  could 
give  rise  to  misleading  recommendations  (and  follow-on  decisions  in  the  worst  case) 
[Banks  et  al.,  2010].  The  challenge  in  this  research  lies  in  the  fact  that  real-world  data 
were  not  available;  from  failure  data  and  baseline  sensor  qualities  of  a  generic  UAS  to 
characteristics  of  an  envisioned  ISHM  suite,  in  terms  of  diagnostics  and  prognostics 
accuracy.  Hence,  various  assumptions  were  made,  in  tenns  of  the  input  parameters,  in 
order  to  derive  useful  output  measures  within  reasonable  boundaries.  Table  9  presents  the 
input  parameters  for  the  discrete-event  simulation  model. 


1  Accessed  on  2  August  2013.  Student  Version  14.5  downloaded  from  Rockwell  Automation,  Inc.  - 
Arena®  website:  http://www.arenasimulation.com/Private  Content.aspx?code=727I57H9K24&type=l 
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Table  9  -  Simulation  Model  Input 


UAS  Properties 
(Input  Parameters) 

Implementation  Remarks 

Typical  Values  for 
Analytic  Considerations 

Failure  Rate,  X 

The  failure  rate  is  the  rate  at  which  failures  occur 
in  a  specified  time  interval. 

number  of  failures 
total  operating  hours 

Assumed  to  be  relatively  constant  during  normal 
UAS  operation  if  system  design  is  mature 
[Blanchard,  2004].  The  failure  rate  is  usually 
what  is  presented  in  FMECA  reports,  e.g.  1 
failure  in  103  hours  (or  X  =  10"'  failure  per  hour). 

*  Based  on  suggested 
probability’  levels  defined  in 
Table  2  fboD.  20001. 

Typical  Values  considered: 
10'1,  10'',  1  O'3  (baseline), 

10'4,  and  10'5 

Unit:  failure  per  hour. 

P(Failure) 

Probability  of  failure  for  a  given  sortie.  Assumed 
to  follow  an  exponential  function  with  constant 
failure  rate,  X  -  calculated  by  the  following 
formula: 

P  {Failure')  —  1  —  Q~A.*SortieFlightTime 

Dependent  on  failure  rate. 

P(Failure  is 
Catastrophic) 

Based  on  NASA  report  in  2007,  there  are  26 
potentially  catastrophic  failures  out  of  132 
identified  failure  conditions  for  a  generic  UAS 
[Hayhurst  et  al.,  2007]. 

Fixed  Parameter 

26 

=  —  =  0.197 

132 

Latency  Threshold 

The  accumulated  latency  effect  that  determines 
criticality  of  a  failure  condition  every  time  it  is 
not  detected  in  previous  flights.  A  latent  failure 
that  reaches  its  pre-determined  Latency  Threshold 
due  to  prior  missed  detections  will  be  upgraded  to 
a  catastrophic  failure  resulting  in  a  mishap. 

Fixed  Parameter  =  5 
(Assumed  that  5  prior 
missed  detections  of  a  non- 
catastrophic  failure  will 
escalate  its  severity  and 
result  in  a  catastrophic 
failure) 

P(Sensor  Detection) 

Assumed  to  be  a  baseline  sensor  characteristic  for 
generic  UAS.  Has  a  positive  correlation  with 
expected  UAS  lifetime. 

Typical  Values  considered: 
0.6,  0.7,  0.8  (baseline),  and 
0.9 

(Assumed  to  be  constant 
over  the  lifetime  of  the  UAS 
without  sensor  degradation) 

P(Sensor  False  Alarm) 

Assumed  to  be  a  baseline  sensor  characteristic  for 
generic  UAS.  Has  a  positive  correlation  with 
unscheduled  sensor  maintenance  actions. 

Typical  Values  considered: 
0.01,  0.02,  0.03  (baseline), 
0.04  and  0.05 

(Assumed  to  be  constant 
over  the  lifetime  of  the  UAS 
without  sensor  degradation) 

Sensor  Missed 
Detection  Threshold 

Assumed  to  be  an  arbitrary  target  threshold  to 
carry  out  sensor  maintenance  when  exceeded; 
should  be  lower  than  assumed  accumulated 
latency  effect  to  be  meaningful. 

Fixed  Parameter  =  4 
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Sensor  False  Alarm 
Threshold 

Assumed  to  be  an  arbitrary  target  threshold  to 
carry  out  sensor  maintenance  when  exceeded; 
should  be  higher  that  missed  detection  threshold 
due  to  no  safety  of  flight  concern. 

Fixed  Parameter  =  10 

Scheduled 

Maintenance  Interval 

Only  applicable  for  Non-ISHM  UAS,  assumed  to 
be  a  fixed  interval  throughout  the  life  of  the  UAS. 

Fixed  Parameter  =  1000 
operating  hours 

Sortie  Flight  Time 
(Normal) 

Assumed  to  be  constant. 

Fixed  Parameter  =  10 

hours 

Sortie  Flight  Time 
(Reduced) 

Assumed  flight  time  to  be  reduced  by  50%  for  a 
Code  3  mission. 

Fixed  Parameter  =  0.5  * 
Normal  Sortie  Flight  Time 
=  5  hours 

ISHM  Properties 
(Input  Parameters) 

Implementation  Remarks 

Fixed  Parameter  / 
Variable? 

P(ISHM  Confidence) 

The  probability  of  ISHM  declaring  that  a  failure 
condition  has  occurred  when  an  actual  failure 
condition  exists  -  an  indication  of  the  strength  of 
the  diagnostic  and  prognostic  algorithms. 
Performance  specification  of  ISHM  system.  Has  a 
positive  correlation  with  the  expected  UAS 
lifetime. 

Typical  Values  considered: 
0.6,  0.7,  0.8  (baseline),  and 
0.9 

(Assumed  to  be  constant 
over  the  lifetime  of  the  UAS 
without  ISHM  degradation) 

P(ISHM  False  Alarm) 

The  probability  of  ISHM  declaring  that  a  failure 
condition  has  occurred  when  there  is  no  actual 
failure.  Performance  specification  of  ISHM 
system.  Has  a  positive  correlation  with 
unscheduled  ISHM  maintenance  actions. 

Typical  Values  considered: 
0.01,  0.02,  0.03  (baseline), 
0.04  and  0.05 

(Assumed  to  be  constant 
over  the  lifetime  of  the  UAS 
without  ISHM  degradation) 

ISHM  Missed 
Detection  Threshold 

Assumed  to  be  an  arbitrary  target  threshold  to 
carry  out  ISHM  maintenance  when  exceeded; 
should  be  lower  than  assumed  accumulated 
latency  effect  to  be  meaningful. 

Fixed  Parameter  =  4 

ISHM  False  Alarm 
Threshold 

Assumed  to  be  an  arbitrary  target  threshold  to 
carry  out  ISHM  maintenance  when  exceeded; 
should  be  higher  that  missed  detection  threshold 
due  to  no  safety  of  flight  concern. 

Fixed  Parameter  =10 

4.2.1  Assumptions 


•  For  an  ISHM-enabled  UAS,  the  detection  outcome  by  the  baseline  sensor  and  the 
eventual  declaration  by  ISHM  were  assumed  to  be  independent  events,  with  the 
latter  being  the  final  authority  in  failure  declaration.  However,  in  a  realistic  ISHM 
implementation  scenario,  the  declaration  results  of  ISHM  will  be  dependent  upon, 
in  part,  the  detection  outcome  of  the  baseline  sensor(s).  The  present  assumption 
was  made  due  to  modeling  limitations  in  conflict  resolution  techniques  between 
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the  baseline  sensor  and  ISHM;  relaxation  of  this  assumption  presents  a  potential 
area  for  future  research. 

•  The  probability  of  failure  for  the  UAS  was  assumed  to  follow  a  constant  failure 
rate.  Likewise,  the  baseline  sensor  and  ISHM  were  modeled  to  exhibit  constant 
probabilities  of  missed  detection  and  false  alarm.  Possible  improvements  to  the 
model  might  consider  failure  distributions  with  increasing  failure  rate  exhibiting 
component  wear-out,  and  also  sensor  and  ISHM  degradation  with  time. 

•  In  typical  FMECA  designs,  failure  conditions  with  greater  severity  will  be 
mitigated  with  redundancy  or  better  detection  devices.  However,  this  model 
assumed  fixed  probabilities  of  sensor  detection  and  ISHM  confidence  regardless 
of  the  severity  of  the  failure  condition. 
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4.3  Model  Implementation 

Based  on  the  analytic  architecture  presented  in  Chapter  III,  two  separate  Arena  ® 
simulation  models  were  set  up;  one  for  an  ISHM-enabled  UAS  and  another  for  a  non- 
ISHM  UAS.  The  sequenced  actions  for  the  two  top-level  models  are  described  in  the 
following  sub-sections.  The  details  of  the  sub-models  implemented  within  Arena  ®  are 
also  provided  in  Appendix  A. 

4.3.1  ISHM-Enabled  UAS  Simulation  Model 

Figure  19  depicts  the  simulation  model  path  if  a  failure  condition  is  present,  while  Figure 
20  refers  to  the  simulation  model  path  if  there  is  no  failure. 

4.3. 1.1  Sequenced  Actions  for  ISHM-Enabled  UAS  Simulation  Model 

•  Sorties  are  generated  as  model  entities  -  with  prc-dclincd  average  sortie  duration. 

•  Probability  of  failure  is  calculated  based  on  occurrence  rate. 

•  The  simulation  perfonns  a  random  draw  from  the  failure  distribution  to  detennine 
the  occurrence  of  failure  for  a  given  sortie. 

•  IF  failure  condition  is  present  (refer  to  Figure  19), 

o  A  1.3  Perform  Failure  Assessment  -  to  determine  if  failure  is  catastrophic 
and  its  latency  effect  (based  on  missed  detections  of  past  failures). 

o  A  1.2  Perfonn  Failure  Detection  -  to  detennine  if  failure  is  detected  by 
baseline  sensor. 

o  A2.1  Perfonn  Diagnostics  and  Prognostics  Assessment  -  to  determine  if 
failure  is  ascertained  by  ISHM. 

o  IF  failure  is  missed  by  ISHM,  detennine  if  it  will  result  in  a  mishap  based 
on  prior  failure  assessment. 

■  IF  mishap  results,  terminate  UAS  life. 
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ELSE  IF  no  mishap  results,  check  status  of  sensor  detection. 


•  IF  failure  is  detected  by  sensor,  a  Code  2  recovery  results 
with  an  ISHM  miss. 

o  A22.1  Perform  Unscheduled  ISHM  Maintenance 
based  on  comparison  with  ISHM  Missed  Detection 
Threshold. 

•  ELSE  IF  failure  is  also  not  detected  by  sensor,  a  Code  1 
recovery  results  with  both  ISHM  and  sensor  misses. 

o  A 13. 3  Assign  Failure  Latency  -  add  latency  effect 
to  UAS  due  to  missed  detection. 

o  ELSE  IF  failure  is  determined  by  ISHM,  a  Code  3  recovery  results  with  an 
unscheduled  component  replacement  action.  Then,  check  status  of  sensor 
detection. 

■  IF  failure  is  missed  by  sensor,  a  Code  3  recovery  results  with  a 
sensor  miss. 

•  A14.3  Perform  Unscheduled  Sensor  Maintenance  based  on 
comparison  with  Sensor  Missed  Detection  Threshold. 

■  ELSE  IF  failure  is  not  missed  by  sensor,  a  Code  3  recovery  results 
with  both  ISHM  and  sensor  confirmations. 

•  ELSE  IF  failure  condition  is  absent  (refer  to  Figure  20), 

o  A  1.2  Perfonn  Failure  Detection  -  to  determine  presence  of  false  alann  by 
baseline  sensor. 

o  A2.1  Perform  Diagnostics  and  Prognostics  Assessment  -  to  determine 
presence  of  false  alarm  by  ISHM. 

o  IF  ISHM  produces  a  false  alarm,  a  Code  3  recovery  results  with  a  Cannot 
Duplicate  (CND). 

■  A22.1  Perform  Unscheduled  ISHM  Maintenance  based  on 
comparison  with  ISHM  False  Alann  Threshold. 

■  IF  sensor  produces  a  false  alann,  a  Code  3  recovery  results  with 
both  ISHM  and  sensor  false  alarms. 
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•  A14.3  Perform  Unscheduled  Sensor  Maintenance  based  on 
comparison  with  Sensor  False  Alarm  Threshold. 

■  ELSE  IF  sensor  does  not  produce  a  false  alarm,  a  Code  3  recovery 
results  with  ISHM  false  alann  only. 

o  ELSE  IF  ISHM  does  not  produce  a  false  alarm,  check  status  of  sensor 
false  alann. 

■  IF  sensor  produces  a  false  alarm,  a  Code  2  recovery  results  with 
sensor  false  alann  only. 

•  A14.3  Perfonn  Unscheduled  Sensor  Maintenance  based  on 
comparison  with  Sensor  False  Alann  Threshold. 

■  ELSE  IF  sensor  does  not  produce  a  false  alarm,  a  Code  1  recovery 
results  -  uneventful  flight  without  false  alarms. 

•  Statistics  are  collected  at  the  end  of  each  sortie  recovery. 

o  UAS  Expected  Lifetime  is  accumulated  through  every  sortie  based  on 
sortie  durations  from  different  recovery  codes. 

o  The  following  categories  of  maintenance  activities  are  added  from 
respective  sorties: 

■  Unscheduled  component  replacement; 

■  Unscheduled  ISHM  maintenance  actions; 

■  Unscheduled  sensor  maintenance  actions. 

•  Simulation  is  tenninated  whenever  a  mishap  occurs. 
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Figure  19  -  Arena  Model:  ISHM-Enabled  UAS  (Failure  Condition 
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Figure  20  -  Arena  Model:  ISHM-Enabled  UAS  (Failure  Condition  Absent) 


4.3.2  Non-ISHM  UAS  Simulation  Model 


Figure  2 1  depicts  the  simulation  model  path  if  a  failure  condition  is  present,  while  Figure 
22  refers  to  the  simulation  model  path  if  there  is  no  failure. 


4.3.2. 1  Sequenced  Actions  for  Non-ISHM  UAS  Simulation  Model 

•  Sorties  are  generated  as  model  entities  -  with  pre-defined  average  sortie  duration. 

•  Probability  of  failure  is  calculated  based  on  occurrence  rate. 

•  Define  scheduled  maintenance  interval. 

•  The  simulation  perfonns  a  random  draw  from  the  failure  distribution  to  determine 
the  occurrence  of  failure  for  a  given  sortie. 

•  IF  failure  condition  is  present  (refer  to  Figure  21), 

o  A  1.3  Perform  Failure  Assessment  -  to  detennine  if  failure  is  catastrophic 
and  its  latency  effect  (based  on  missed  detections  of  past  failures). 

o  A  1.2  Perform  Failure  Detection  -  to  detennine  if  failure  is  detected  by 
baseline  sensor. 

o  IF  failure  is  detected  by  sensor,  a  Code  3  recovery  results  with  an 
unscheduled  component  replacement  action. 

■  Reset  maintenance  schedule  whenever  a  component  replacement 
has  been  earned  out. 

o  ELSE  IF  failure  is  not  detected  by  sensor,  determine  if  it  will  result  in  a 
mishap  based  on  prior  failure  assessment. 

■  IF  mishap  results,  terminate  UAS  life. 

■  ELSE  IF  no  mishap  results,  a  Code  1  recovery  results  with  a  sensor 
miss  (but  this  is  unknown  to  the  UAS). 

•  A 13. 3  Assign  Failure  Latency  -  add  latency  effect  to  UAS 
due  to  missed  detection. 

•  A14.1  Perfonn  Scheduled  Maintenance  based  on 
comparison  with  remaining  time  to  scheduled  maintenance. 
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•  ELSE  IF  failure  condition  is  absent  (refer  to  Figure  22), 

o  A  1.2  Perform  Failure  Detection  -  to  detennine  presence  of  false  alann  by 
baseline  sensor. 

o  IF  sensor  produces  a  false  alann,  a  Code  3  recovery  results  with  a  Cannot 
Duplicate  (CND). 

■  A22.1  Perfonn  Unscheduled  Sensor  Maintenance  based  on 
comparison  with  Sensor  False  Alann  Threshold. 

■  A14.1  Perform  Scheduled  Maintenance  based  on  comparison  with 
remaining  time  to  scheduled  maintenance. 

o  ELSE  IF  sensor  does  not  produce  a  false  alarm,  a  Code  1  recovery  results 
-  uneventful  flight  without  false  alarm. 

■  A14.1  Perform  Scheduled  Maintenance  based  on  comparison  with 
remaining  time  to  scheduled  maintenance. 

•  Statistics  are  collected  at  the  end  of  each  sortie  recovery. 

o  UAS  Expected  Lifetime  is  accumulated  through  every  sortie  based  on 
sortie  durations  from  different  recovery  codes. 

o  The  following  categories  of  maintenance  activities  are  added  from 
respective  sorties: 

■  Unscheduled  component  replacements; 

■  Scheduled  maintenance  actions; 

■  Unscheduled  sensor  maintenance  actions  due  to  false  alarms  only. 

•  Simulation  is  tenninated  whenever  a  mishap  occurs. 
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Figure  21  -  Arena  Model:  Non-ISHM  UAS  (Failure  Condition  Present) 


See  Figure  22 


Figure  22  -  Arena  Model:  Non-ISHM  UAS  (Failure  Condition 
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4.3.3  Verification  and  Validation 


4.3.3. 1  Verification  -  Did  I  Build  the  Model  Correctly? 

Diligent  effort  was  expended  to  prove  the  model  wrong.  Numerous  iterations  were  tested 
until  errors  in  design,  coding  logic,  process  flows,  or  concepts  were  found.  The  following 
analysis  tools  were  used  to  test  the  models  and  deem  them  either  adequate  or  incorrect: 


a)  Software  Debugging  Function  -  A  first-cut  verification  was  conducted  by  using 
the  Arena  ®  debugger  to  ensure  that  no  coding  irregularities  or  warnings  exist  for 
the  simulation  model. 

b)  Stress  Testing  -  There  were  multiple  variables  modeled  with  statistical 
distributions  or  percentage -based  decisions  within  the  model.  These  variables 
were  varied  using  ‘non-realistic’  levels  based  on  experience  to  ensure  logical 
results  would  be  generated. 

c)  Animation  -  This  feature  that  is  available  within  Arena  ®  was  used  to  trace  a 
UAS  sortie  through  the  model.  Furthermore,  animation  was  used  to  ensure  entities 
did  not  enter  infinite  loops  or  proceed  along  ‘illogical  paths  ’. 

d)  Analytical  Method  Using  Probability  Theory  -  Based  on  probability  theory, 
analytical  methods  can  be  employed  to  verify  certain  output  measures  of  the 
model.  A  close  approximation  of  simulation  results  to  analytical  values  provides 
confidence  that  the  model  is  accurate. 


The  following  parameters  were  defined  for  a  test  simulation  run. 

Table  10  -  Test  Values  for  Theoretical  Calculation 


Parameter 

Formula  or 
Label 

Specified  or  Calculated  Value 

Normal  Sortie  Duration 

ASD 

10  hours 

Failure  Occurrence  Rate 

X 

10"6  failure  per  hour 

Probability  of  Failure  Occurring 
in  a  Given  Sortie 

PF  =  i  -  e4-*ASD 

1  _  g-0.000001  *  10  _  q  H= 

Probability  of  Sensor  Detection 

PsD 

0.8 

Probability  of  ISF1M  Confidence 

Pic 

0.8 
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Assuming  that  the  above  decision  processes  are  independent,  probability  of  a 
failure  occurring  and  being  detected  by  both  ISHM  and  baseline  sensor 


-  [PF  fl  Psd  H  Pic] 

=  9.99995  *  10'6  *  0.8  *  0.8  =  6.400  *  10  6 

The  same  test  values  from  Table  10  were  input  for  the  ISHM-enabled  UAS 
Arena®  model.  For  the  test  simulation  run,  the  above  probability  should  be 
approximately  equal  to: 

Code  3  With  Both  ISHM  and  Sensor  Confirmation 
Total  Sorties  Generated 

From  the  results  of  the  simulation  run  over  three  test  replications,  the  average 
values  obtained  are: 


Table  11  -  Results  for  Test  Simulation  Run 


Replication 

Code  3  with 
ISHM  &  Sensor 
Confirmation 

Total  Sorties 
Generated 

Code  3  with  ISHM  &  Sensor  Confirmation 

Total  Sorties  Generated 

#1 

14 

2.4370  *  106 

5.745  *  10'6 

#2 

3 

3.3350  *  10" 

8.996  *  10-6 

#3 

10 

1.5239  *  106 

6.562  *  10-6 

Average  Value: 

7.101  *  10  6 

e)  It  can  be  seen  that  the  values  of  the  theoretical  calculation  and  the  test  simulation 
are  in  agreement  within  an  11%  difference4  -  and  the  difference  is  expected  to 
decrease  with  greater  statistical  accuracy  achieved  through  more  replications. 
Though  not  all  outputs  of  the  simulation  can  be  verified  through  analytical 
methods,  partial  verification  of  such  measures  lends  confidence  and  weight  to  the 
overall  accuracy  of  the  model. 


4  The  actual  percentage  difference  of  10.95%  between  the  theoretical  and  simulation  results  for  the  test  case 
appeared  to  be  magnified  due  to  small  fractions  resulting  from  the  large  numbers  of  sorties  generated 
(denominator)  with  an  extremely  low  failure  occurrence  rate  (10  6).  However,  this  provided  an  extreme  case 
for  our  simulation  boundaries.  If  we  increase  the  failure  occurrence  rate  to  10"3  (which  will  be  more  typical 
of  existing  UAS  designs),  the  percentage  difference  between  the  simulation  and  analytical  results  is 
reduced  to  2.51%. 
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4.3.3.2  Validation  -  Did  I  Build  the  Correct  Model? 


Upon  verification  of  the  model,  the  next  step  of  validation  posed  certain  challenges 
because  no  real-world  or  historical  data  was  available  to  validate  the  output  results  of  the 
model.  Hence,  external  perspectives  were  an  important  source  of  validation.  Consultation 
with  my  thesis  advisor  and  other  students  involved  in  ISHM  research  provided  valuable 
insights  on  process  logic  and  modeling  techniques.  The  final  validation  shall  be  provided 
by  the  research  sponsors  from  AFRL  who  will  be  able  to  provide  Subject  Matter  Expert 
(SME)  advice  on  the  applicability  of  this  research  model. 

4.4  Sensitivity  Analysis 

This  section  discusses  the  test  scenarios  to  achieve  various  ISHM  implementation 
objectives.  Although  a  large  number  of  input  variables  and  possible  values  exist,  the 
design  of  the  various  experiments  fixed  certain  parameters  at  reasonable  baseline  values 
as  defined  in  Table  9.  Thereafter,  variables  of  interest  (factors)  were  varied  at  defined 
levels  and  provided  as  inputs  to  the  simulation  models.  Using  Arena  ®  Process  Analyzer, 
results  (responses)  from  the  simulation  models  were  obtained  and  the  various  scenarios 
were  ranked  in  tenns  of  achievement  of  desired  objectives.  In  order  to  improve  the 
statistical  accuracy  for  the  simulation  results,  each  test  scenario  was  run  for  10 
replications  for  every  experiment.  In  addition,  a  Common  Random  Number  (CRN) 
strategy  was  employed  to  reduce  random  variance  (“noise”)  between  the  ISHM-Enabled 
UAS  and  Non-UAS  models,  i.e.  assigning  the  same  random  number  streams  to  failure 
occurrence  and  detection  decisions  within  the  two  simulation  models  [Banks  et  al, 
2010]. 
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The  first  experiment  demonstrates  the  dependence  of  UAS  expected  lifetime  on  failure 
occurrence  rate.  With  the  boundaries  provided  by  the  first  experiment,  in  tenns  of  failure 
occurrence  rates,  the  second  experiment  investigates  those  relevant  scenarios  with  the 
greatest  percentage  extension  of  UAS  life  expectancy.  The  third  and  fourth  experiments 
seek  to  establish  a  business  case  for  ISHM  implementation  through  reduction  of  Cannot- 
Duplicate  (CND)  cases  and  maintenance  activities. 


4.4.1  Dependence  of  UAS  Expected  Lifetime  on  Failure  Occurrence  Rate,  k 

This  experiment  seeks  to  identify  those  parameters  that  have  an  influence  on  UAS 
expected  lifetime.  Table  12  lists  the  various  factors  and  levels  considered  for  this 
experiment.  Based  on  the  three  factors  and  their  associated  levels,  a  total  of  80  scenarios 
are  being  tested  for  the  desired  response.  Appendix  B:  Table  B1  shows  the  results  of  this 
experiment  in  terms  of  the  relative  ranking  of  the  80  scenarios  in  achieving  the  longest 
UAS  expected  lifetime  of  an  ISHM-enabled  UAS. 


Table  12  -  Experiment  1:  Design  Factors  and  Levels 


Input/Control  Variables 

Defined  Values 

Factors 

Levels 

Failure  Rate,  X 

Iff1,  Iff2,  Iff3,  Iff4,  Iff 5 

Unit:  failure  per  hour. 

P(  Sensor  Detection) 

0.6,  0.7,  0.8,  0.9 

P(ISHM  Confidence) 

0.6,  0.7,  0.8,  0.9 

Output  Performance  Measure 

(Response) 

UAS  Expected  Lifetime 

Unit:  Hours 
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Results:  The  experiment  showed  that  the  failure  occurrence  rate  has  the  greatest  effect  on 
the  UAS  expected  lifetime.  Figure  23  illustrates  the  range  of  UAS  lifetimes  associated 
with  the  various  levels  of  failure  occurrence  rates  defined  for  the  experiment.  For  an 
extremely  low  failure  occurrence  rate  of  1 0"5,  it  is  possible  to  achieve  over  a  million  flight 
hours  for  an  UAS  with  or  without  ISHM  implementation.  However,  UAS  are  not 
typically  designed  for  such  extended  lifetimes,  as  age-related  issues  such  as  structural 
health  will  outweigh  the  benefits  provided  by  ISHM.  On  the  other  hand,  a  high  failure 
occurrence  rate  of  10'1  will  reduce  the  expected  lifetime  to  a  few  hundred  flying  hours  - 
and  such  a  low-reliability  UAS  will  not  be  cost-effective  for  ISHM  consideration. 
Hence,  this  experiment  tightened  the  bounds  for  failure  occurrence  rates  in  subsequent 
experiments.  Thus,  only  failure  rates  between  10"4  and  10'2  per  flying  hour  will  be 
considered  in  the  remaining  experiments. 


Failure  Occurrence  UAS  Lifetime 

Rate  (ISHM-Enabled  UAS) 

[Per  Flying  Hour]  [Hours] 

'=— P  3,662,213.50 


A  =  0.00001 


X  =  0.0001 


A  =  0.001 


A  =  0.01 


1.541.751.50 

354.312.50 

139,169.00 

38,676.00 

16.477.50 

3.878.50 

1,689.00 

432.50 


A  =  0.1 


185.00 


Figure  23  -  Experiment  1:  Dependence  of  UAS  Expected  Lifetime  on  Failure  Occurrence  Rate 

74 


4.4.2  Maximizing  Percentage  of  Lifetime  Extension 


This  experiment  seeks  to  identify  those  scenarios  that  will  yield  the  greatest  benefit  in 
tenns  of  operational  lifetime  extension  through  ISHM  implementation.  With  the  revised 
bounds  for  failure  occurrence  rates,  the  input  factors/levels  are  shown  in  to  Table  13,  with 
48  scenarios  being  tested  for  the  desired  response.  Appendix  B:  Table  B-2  shows  the 
results  of  this  experiment  in  terms  of  the  relative  ranking  of  the  48  scenarios  in  achieving 
the  greatest  percentage  of  lifetime  extension  through  ISHM  implementation. 


Table  13  -  Experiment  2:  Design  Factors  and  Levels 


Input/Control  Variables 

Factors 

Defined  Values 

Levels 

Failure  Rate,  X 

m2,  iff3,  iff4 

Unit:  failure  per  hour. 

P(  Sensor  Detection) 

0.6,  0.7,  0.8,  0.9 

P(ISHM  Confidence) 

0.6,  0.7,  0.8,  0.9 

Output  Performance  Measure 
(Response) 

Percentage  Gain  in  Lifetime  Extension  (%) 

Results:  From  the  simulation  results,  it  was  observed  that,  for  a  non-ISHM  UAS,  the 
expected  lifetime  is  highly  sensitive  to  its  baseline  sensor  probability  of  detection. 
However,  for  an  ISHM-enabled  UAS,  in  scenarios  where  the  probability  of  ISHM 
confidence  is  higher  than  the  baseline  sensor  probability  of  detection,  the  expected 
lifetime  is  relatively  insensitive  to  the  latter,  but  highly  sensitive  to  the  fonner.  The 
reason  for  the  latter  is  because  in  the  ISHM-enabled  UAS  model,  the  eventual  declaration 
by  ISHM  of  a  failure  will  override  the  detection  result  of  the  baseline  sensor. 
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Therefore,  to  achieve  the  most  benefit  through  ISHM  implementation,  in  terms  of 
lifetime  extension,  the  probability  of  ISHM  confidence  must  be  much  higher  than  the 
probability  of  baseline  sensor  detection. 

For  scenarios  with  the  same  probabilities  of  baseline  sensor  detection  and  ISHM 
confidence,  the  deciding  factor  appeared  to  be  on  the  former.  Refer  to  Table  14.  With  the 
baseline  sensor  having  a  probability  of  detection  above  0.8,  the  experiment  showed  that 
implementing  ISHM  does  not  improve  the  UAS  expected  lifetime. 


Table  14  -  Experiment  2:  Scenarios  with  Same  Probabilities  of  Baseline  Sensor  Detection 

and  ISHM  Confidence 


Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of 
Detection 
(Baseline  Sensor) 

Probability  of  ISHM 
Confidence 

UAS  Lifetime 
(ISHM-Enabled  UAS) 
[Hours] 

(a) 

UAS  Lifetime 
(Non-ISHM  UAS) 
[Hours] 

(b) 

Increase  in  UAS 

Lifetime 
(c)  =  (a)-(b) 

%  Lifetime  Gain 
(d)  =  (c)/(b) 

1 

0.001 

0.6 

0.6 

16477.5 

10755.5 

5722 

53.20% 

2 

0.01 

0.6 

0.6 

1689 

1105 

584 

52.85% 

3 

0.0001 

0.6 

0.6 

139169 

103479.5 

35689.5 

34.49% 

4 

0.001 

0.7 

0.7 

18828 

15623 

3205 

20.51% 

5 

0.01 

0.7 

0.7 

1914.5 

1687.5 

227 

13.45% 

6 

0.0001 

0.7 

0.7 

158851.5 

154323.5 

4528 

2.93% 

7 

0.001 

0.8 

0.8 

22096.5 

24979.5 

-2883 

-11.54% 

8 

0.01 

0.8 

0.8 

2266 

2634.5 

-368.5 

-13.99% 

9 

0.0001 

0.8 

0.8 

191844.5 

253859 

-62014.5 

-24.43% 

10 

0.001 

0.9 

0.9 

38676 

74975.5 

-36299.5 

-48.42% 

11 

0.01 

0.9 

0.9 

3878.5 

7679.5 

-3801 

-49.50% 

12 

0.0001 

0.9 

0.9 

354312.5 

731245.5 

-376933 

-51.55% 

4.4.3  Minimizing  Number  of  Cannot-Duplicate  (CND)  Cases 

This  experiment  seeks  to  identify  those  scenarios  that  will  reduce  troubleshooting  times 
for  maintainers  in  dealing  with  Cannot-Duplicate  (CND)  defect  reports.  The  benefits  of 
reduced  CND  cases  can  be  further  quantified  in  terms  of  reduced  maintenance  time 
leading  to  better  availability  of  the  UAS.  In  addition,  conservative  maintenance  policies 
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that  dictate  the  replacement  of  component  for  precautionary  measures  in  the  event  of 
CND  will  increase  maintenance  costs  significantly.  In  the  absence  of  a  failure  condition, 
the  design  specifications  of  baseline  sensors  and  ISHM  (in  terms  of  probabilities  of  false 
alann)  will  detennine  the  number  of  CND  cases.  Table  15  lists  the  various  factors  and 
levels  considered  for  this  experiment  and  a  total  of  75  scenarios  are  being  tested  for  the 
desired  response.  Appendix  B:  Table  B3  shows  the  results  of  this  experiment  in  tenns  of 
the  relative  ranking  of  the  75  scenarios  in  achieving  the  maximum  reduction  in  terms  of 
CND  cases. 


Table  15  -  Experiment  3:  Design  Factors  and  Levels 


Input/Control  Variables 

Factors 

Defined  Values 

Levels 

Failure  Rate,  X 

Iff2,  Iff3,  Iff4 

Unit:  failure  per  hour. 

P(  Sensor  False  Alarm) 

0.01,  0.02,  0.03,  0.04,  0.05 

P(ISHM  False  Alarm) 

0.01,  0.02,  0.03,  0.04,  0.05 

Output  Performance  Measure 
(Response) 

Reduction  in  CND  Cases 

Results:  From  the  simulation  results,  it  was  observed  that  for  a  non-ISHM  UAS,  the 
number  of  CND  cases  is  highly  sensitive  to  its  baseline  sensor  probability  of  false  alann. 
However,  for  an  ISHM-enabled  UAS,  in  scenarios  where  the  probability  of  ISHM  false 
alann  is  lower  than  the  baseline  sensor  probability  of  false  alarm,  the  number  of  CND 
cases  is  relatively  insensitive  to  the  latter,  but  highly  sensitive  to  the  former.  The  reason 
for  the  latter  is  because  in  the  ISHM-enabled  UAS  model,  the  eventual  declaration  by 


ISHM  of  a  non-failure  will  override  the  detection  result  of  the  baseline  sensor.  This  result 
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can  be  illustrated  by  Figure  24.  Considering  the  specific  case  when  the  failure  rate  is 
0.001  per  flying  hour,  the  surface  plot  showed  that  there  is  positive  reduction  in  CND 
cases  only  when  the  probability  of  the  baseline  sensor  false  alarm  is  higher  than  or  equals 
to  the  probability  of  ISHM  false  alarm  (as  represented  by  the  upper  light  blue  area). 


Percentage  Reduction  in  CND  Cases 


Failure  Occurrence  Rate  =  0.001 


■  400%  300%  ■  300%  200%  ■ -200%  100%  ■  100%  0%  «0%100% 


Figure  24  -  Experiment  3:  Minimizing  Number  of  CND  Cases 


From  the  analytic  model,  higher  CND  counts  (over  an  UAS’  lifetime)  are  expected  for 
more  reliable  UAS.  To  achieve  a  greater  reduction  in  the  number  of  CND  cases,  the 
probability  of  ISFIM  false  alarm  must  be  much  lower  than  the  probability  of  false  alarm 
for  the  baseline  sensor. 

Refer  to  Table  16.  For  scenarios  where  the  probabilities  of  false  alarm  are  the  same  for 
the  baseline  sensor  and  ISFIM,  the  simulation  results  suggested  that  ISHM 
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implementation  is  only  effective  in  reducing  CND  cases  when  the  failure  occurrence  rate 
is  less  than  10" . 


Table  16  -  Experiment  3:  Scenarios  with  Same  Probabilities  of  Baseline  Sensor  False  Alarm  and 

ISHM  False  Alarm 


Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of  False 

Alarm 

(Baseline  Sensor) 

Probability  of 
ISHM  False  Alarm 

Total  Cases  of  CND 
(ISHM-Enabled  UAS) 
[a] 

Total  Cases  of  CND 
(Non-ISHM  UAS) 

[b] 

Reduction  in  CND 
[c]=[b]-[a] 

%  Reduction  in 

CND 

[d]=[c]/[b] 

1 

0.0001 

0.05 

0.05 

977 

1286 

309 

24.03% 

2 

0.0001 

0.04 

0.04 

778 

1023 

245 

23.95% 

3 

0.0001 

0.03 

0.03 

584 

768 

184 

23.96% 

4 

0.0001 

0.02 

0.02 

390 

515 

125 

24.27% 

5 

0.0001 

0.01 

0.01 

193 

252 

59 

23.41% 

6 

0.001 

0.05 

0.05 

111 

128 

17 

13.28% 

7 

0.001 

0.04 

0.04 

89 

103 

14 

13.59% 

8 

0.001 

0.03 

0.03 

66 

76 

10 

13.16% 

9 

0.001 

0.02 

0.02 

45 

51 

6 

11.76% 

10 

0.001 

0.01 

0.01 

22 

25 

3 

12.00% 

11 

0.01 

0.05 

0.05 

11 

13 

2 

15.38% 

12 

0.01 

0.03 

0.03 

7 

8 

1 

12.50% 

13 

0.01 

0.04 

0.04 

9 

10 

1 

10.00% 

14 

0.01 

0.01 

0.01 

2 

2 

0 

0.00% 

15 

0.01 

0.02 

0.02 

5 

5 

0 

0.00% 

4.4.4  Minimizing  Number  of  Maintenance  Actions 

This  experiment  seeks  to  identify  those  scenarios  that  will  reduce  the  overall  maintenance 
actions.  It  is  assumed  in  the  analytic  model  that  an  ISHM-enabled  UAS  will  not  require 
scheduled  maintenance  due  to  its  added  onboard  diagnostics  and  prognostics  capabilities. 
However,  additional  maintenance  for  an  ISHM-enabled  UAS  will  be  driven  by  generated 
ISHM  false  alarms.  The  benefits  of  reduced  maintenance  actions  can  be  further 
quantified  by  assigning  a  cost  factor  to  each  of  the  maintenance  actions.  This  will  provide 
justifications  in  relaxing  maintenance  schedule  intervals  and  eventually  a  condition-based 
maintenance  program  through  ISHM.  Table  17  lists  the  various  factors  and  levels 


considered  for  this  experiment.  Similar  to  the  previous  experiment,  a  total  of  75  scenarios 
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are  being  tested  for  the  desired  response.  Appendix  B:  Table  B4  shows  the  results  of  this 
experiment  in  terms  of  the  relative  ranking  of  the  75  scenarios  in  achieving  the  maximum 
reduction  in  terms  of  maintenance  actions. 


For  ISHM-Enabled  UAS,  total  maintenance  actions  include: 


•  Total  Number  of  CND  Cases  (troubleshooting  required) 

•  Total  Number  of  Unscheduled  ISHM  Maintenance  (due  to  ISHM  missed 
detections  and  false  alarms) 

•  Total  Number  of  Unscheduled  Sensor  Maintenance  (due  to  baseline  sensor 
missed  detections  and  false  alarms) 


For  Non-ISHM  UAS,  total  maintenance  actions  include: 


•  Total  Number  of  CND  Cases  (troubleshooting  required) 

•  Total  Number  of  Scheduled  Maintenance  (with  maintenance  interval  of 
1000 flying  hours) 

•  Total  Number  of  Unscheduled  Sensor  Maintenance  (due  to  baseline  sensor 
false  alarms  only) 


Table  17  -  Experiment  4:  Design  Factors  and  Levels 


Input/Control  Variables 

Factors 

Defined  Values 

Levels 

Failure  Rate,  X 

iff2,  iff3,  iff4 

Unit:  failure  per  hour. 

P(Sensor  False  Alarm) 

0.01,  0.02,  0.03,  0.04,  0.05 

P(ISHM  False  Alarm) 

0.01,  0.02,  0.03,  0.04,  0.05 

Output  Performance  Measure 

(Response) 

Reduction  in  Maintenance  Actions 
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Results:  Of  the  75  tested  scenarios,  a  total  of  46  scenarios  showed  a  reduction  in  overall 


maintenance  activities  with  ISHM  implementation.  As  an  illustration,  refer  to  Figure  25 
for  the  specific  case  when  the  failure  occurrence  rate  is  0.01  per  flying  hour.  If  we 
consider  the  effects  of  probabilities  of  false  alarm  on  the  number  of  maintenance  actions, 
it  was  observed  from  this  surface  plot  that  there  is  positive  reduction  in  maintenance 
actions  only  when  the  probability  of  the  baseline  sensor  false  alarm  is  higher  than  the 
probability  of  ISFIM  false  alarm  (as  represented  by  the  upper  light  blue  area). 


■  400%  300%  ■  300%  200%  ■  200%  100%  ■  100%  0%  «0%100% 


-200% 


Percentage  Reduction  in 
Maintenance  Actions 

Failure  Occurrence  Rate  =  0.01 


100% 

0% 


0.05 


-300% 

400% 


Figure  25  -  Experiment  4:  Minimizing  Number  of  Maintenance  Actions 


From  a  system  engineer’s  perspective,  a  cost-benefit  analysis  can  be  derived  from  the 
results  of  this  experiment.  Each  response  value  in  terms  of  reduction  in  maintenance 
actions  can  be  assigned  a  cost  value.  The  expected  cost  savings  can  then  be  weighed 
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against  the  design  budget  or  the  expected  cost  of  the  ISHM  system  (with  specified 
perfonnance  requirements)  to  evaluate  its  cost-effectiveness. 

4.5  Analysis  Summary 

This  chapter  provided  details  of  the  Arena  ®  simulation  and  highlighted  the  assumptions 
in  input  modeling  due  to  data  inadequacies.  Thereafter,  four  experiments  were  set-up  to 
test  full-factorial  scenarios  in  achieving  desired  ISHM  perfonnance  measures.  The 
desired  outcome  of  the  validated  scenarios  is  to  provide  future  UAS  designers  with  an 
analytical  tool  to  identify  perfonnance  specifications  of  baseline  UAS  and 
complementing  ISHM  systems  to  achieve  a  desired  UAS  lifetime  extension  or  reduction 
in  maintenance  costs. 
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V.  Conclusions  and  Recommendations 


5.1  Chapter  Overview 

This  chapter  will  discuss  the  answers  to  the  research  objectives  and  recommend  areas  for 
future  research. 

5.2  Research  Questions  Answered 

The  focus  of  this  research  was  to  quantify  the  continued  airworthiness  benefits  of  ISHM 
by  developing  an  analytic  architecture  for  comparison  between  an  “as-is”  UAS  (without 
ISHM  capabilities)  and  a  “to-be”  UAS  (with  ISHM  capabilities).  From  an  airworthiness 
certification  perspective,  this  research  sought  to  develop  a  method  for  establishing 
performance  requirements  for  components  of  an  ISHM-enabled  UAS. 

Prior  to  developing  the  architecture,  a  literature  review  was  conducted  to  gain  a  better 
understanding  of  the  research  arena.  The  following  questions  were  posed  in  Chapter  I  for 
literature  review: 

(1)  What  is  system  health  monitoring/management  and  what  are  some  related 
applications? 

There  are  many  different  terms  and  associated  definitions  on  system  health  management 
available  in  the  literature  and  this  research  chose  the  term,  Integrated  System  Health 
Management  (ISHM)  -  aligned  to  previous  research  in  the  department  [Stonn,  2013]. 
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Benedettini  et  al.  [2009]  termed  system  health  management  as  integrated  vehicle  health 
management  (IVHM)  and  defined  it  as  “a  collection  of  data  relevant  to  the  present  and 
future  performance  of  a  vehicle  system  and  its  transfonnation  into  information  can  be 
used  to  support  operational  decisions”.  The  authors  also  presented  various  definitions 
found  in  the  literature  that  sought  to  provide  a  top-level  perspective  of  ISHM. 

In  terms  of  related  ISHM  applications,  Health  and  Usage  Monitoring  Systems  (HUMS) 
surfaced  as  one  of  the  more  matured  applications  related  to  airworthiness  of  helicopters. 
These  systems  emerged  in  the  1980s  as  a  response  to  the  high  accident  rates  experienced 
by  offshore  shuttle  helicopters  traversing  the  petrol  installations  in  the  North  Sea,  and 
increased  demand  for  improved  operational  safety  and  reduced  rotorcraft  maintenance 
costs  had  paved  the  way  for  HUMS  [Wiig,  2006].  In  tenns  of  process  management,  the 
typical  processes  of  a  HUMS  program  are  previously  depicted  in  Figure  2  [JHSIT,  2013]. 
Basic  HUMS  operations  include  real-time  data  collection  and  download,  defining 
thresholds  based  on  trending  information,  and  comparison  of  health  indicators  against 
these  thresholds  to  ensure  continued  airworthiness  of  the  aircraft.  This  concept  of 
operations  is  definitely  in  agreement  with  that  of  a  typical  ISHM  implementation.  HUMS 
originated  from  an  airworthiness  concern,  and  its  implementation  drove  extensive 
research  in  condition-based  rotorcraft  maintenance.  Understanding  the  HUMS 
architecture  and  associated  processes  provided  alignment  in  this  research  in  developing 
the  analytic  architecture  and  associated  maintenance  cost  models. 
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Apart  from  aerospace  applications,  the  literature  review  also  cited  a  technological 
breakthrough  from  the  health  care  industry  -  to  draw  parallel  inferences  of  ISHM  from 
another  systems  thinking  perspective.  Artificial  intelligence  in  the  form  of  IBM’s  Watson 
provides  consistency  of  decisions  amongst  available  medical  solutions,  based  on  accurate 
clinical  examinations  and  evidence  [Cohn,  2013],  Such  technological  breakthroughs  in 
health  care,  in  the  areas  of  sensor  data  fusion  and  artificial  intelligence,  presented 
similarities  to  an  ideal  ISHM  architecture  that  integrates  processed  sensor  information 
and  intelligence  through  diagnostics  and  prognostics  algorithms. 

(2)  What  are  the  essential  elements  of  ISHM? 

The  ISHM  design  and  operation  concept  embraces  an  integration  of  sensors, 
communication  technologies,  and  artificial  intelligence  to  provide  vehicle-wide  abilities 
to  diagnose  problems  and  recommend  solutions  [Benedettini  et  al,  2009].  At  the  front 
end  of  a  typical  ISHM  architecture  is  a  sensor  suite  responsible  for  gathering  state 
awareness  variables  that  are  indicative  of  potential  failure  modes.  For  an  envisaged 
ISHM  configuration,  apart  from  conventional  sensors  that  monitor  and  control  sub¬ 
systems,  system-level  sensor  suites  are  also  being  introduced  in  the  fonn  of  smart 
embedded  sensor  systems  with  wireless  communications  transfer  protocol  in  place  for 
overall  system  health  management.  Upon  filtering  of  sensor  data  to  extract  relevant  fault 
features,  the  diagnostics  module  analyzes  the  fault  features  to  detect,  identify  and  isolate 
impending  fault  conditions.  In  addition,  with  health  and  usage  data  being  fed  to  the 
prognostic  module,  the  latter  is  able  to  combine  historical  data  to  generate  an  estimation 

of  the  time-to-failure  of  specific  subsystems  and  components.  Depending  on  the  level  of 
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autonomy,  such  diagnostic  and  prognostic  information  can  be  processed  on-board  the 
vehicle  through  its  auto-recovery  systems  or  communicated  to  technical  support 
managers  on  ground.  In  order  to  highlight  the  similarities  in  considerations  between  the 
analytic  architecture  developed  in  this  research  and  the  typical  ISHM  architecture 
described  in  the  literature  review,  the  relevant  operational  activities  from  the  OV-5a  were 
superimposed  on  Figure  3  that  previously  depicted  the  suggested  architecture  for  ISHM. 


A11.1  Aviate 


Figure  3  -  Typical  ISHM  Architecture  |Benedettini  et  al.,  2009] 

(*  Superimposed  with  operational  activities  of  developed  analytic  architecture) 


(3)  What  are  the  critical  FMECA  hazards  associated  with  a  typical  UAS  and  their 
relation  to  airworthiness? 

NASA  published  a  report  in  2007  that  identified  the  typical  failure  conditions  of  a  UAS 
based  on  functional  decomposition  of  a  generic  UAS  [Hayhurst  et  al.,  2007].  The  full 
functional  decomposition  is  relatively  large,  with  69  functions  at  the  lowest  level  under 
the  major  functions  of  aviate,  navigate,  communicate  and  mitigate.  The  primary  safety 
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goal  of  the  failure  hazard  assessment  is  to  avoid  any  UAS-initiated  decrease  in  the  safety 
of  the  National  Air  Space  (NAS).  As  a  result,  failure  condition  criticality  is  determined 
by  its  effect  on  people  on  ground  or  in  other  aircraft.  Damage  to  material  assets  is  out-of- 
scope,  unless  it  affects  human  safety.  The  majority  of  potential  failure  conditions  fall 
under  the  Aviate  or  Mitigate  functions.  In  the  assessment  by  NASA,  twenty-six 
potentially  catastrophic  failure  conditions  out  of  132  were  identified,  considering  only 
single  failures  in  the  en-route  phase  of  flight. 

Although  severity  effects  related  to  the  occupants  of  an  aircraft  do  not  apply  to  unmanned 
systems,  other  severe  outcomes  are  possible  that  will  result  in  human  casualties,  either  in 
other  aircraft  or  on  the  ground.  Ground  impact  can  endanger  the  general  public,  and  midair 
collision  with  a  manned  aircraft  can  threaten  the  safety  of  the  passengers  aboard  that  aircraft. 
Both  effects  are  critical  system  design  drivers  that  have  implications  for  UAS  operations  and 
reliability  requirement.  As  the  size  of  an  UAS  grow  from  Micro  — >  Mini  — >  Tactical  — > 
MALE  (Medium  Altitude  Long  Endurance)  — >  HALE  (High  Altitude  Long  Endurance)  — > 
Heavy  classifications  in  terms  of  weight,  there  will  be  an  increased  level  of  hazard  risk 
involved  in  the  operation  of  UAS  [Weibel  and  Hansman,  2005]. 

Airworthiness  requirements  and  certification  for  UAS  are  hence  crucial  in  assessing  its 
inherent  operating  risk.  System  safety  is  the  application  of  engineering  and  management 
principles  (such  as  FMECA),  criteria  and  techniques  to  optimize  all  aspects  of  safety 
within  the  constraints  of  operational  effectiveness,  time  and  cost  throughout  all  phases  of 
the  system  life  cycle.  A  system  safety  program  is  crucial  for  the  initial  and  continued 
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airworthiness  of  all  weapon  systems,  both  manned  and  unmanned.  Understanding  the 
functional  decomposition  of  a  generic  UAS  and  the  potential  failure  conditions  and 
effects  provided  an  appreciation  of  the  top-level  functional  failure  modes  in  architecting 
ISHMfor  initial  and  continued  airworthiness. 

(4)  What  is  Condition-Based  Maintenance  (CBM)? 

Though  the  benefits  of  ISHM  are  well  recognized,  the  most  quantifiable  benefits  in  the 
current  paradigm  were  almost  exclusively  reported  in  terms  of  maintenance  related 
savings.  Specifically,  HUMS  and  CBM  are  almost  synonymous  in  the  rotorcraft  world. 
CBM,  also  known  as  unscheduled/corrective  maintenance,  defines  a  maintenance  policy 
where  components  are  replaced  upon  detection  of  an  unsatisfactory  condition  -  such  as 
those  detected  (or  even  predicted)  by  an  ISHM  system.  Every  component  has  a  safe  life 
or  operating  life,  beyond  which  continued  operation  of  the  component  could  result  in 
catastrophic  failures.  In  a  scheduled  maintenance  policy,  there  is  potential  for  wastage 
should  a  component  be  replaced  way  ahead  of  its  life  expiration.  With  the  potential  of 
increased  health  management  and  surveillance  presented  by  ISHM,  the  useful  life  of  a 
component  can  be  extended  until  diagnostic  or  prognostic  algorithms  decide  that  an 
impending  failure  is  imminent.  To  implement  a  safe  and  effective  CBM  program,  it 
should  be  cautioned  that  the  algorithms  will  have  to  consider  the  uncertainty  and 
confidence  levels  regarding  a  component’s  useful  life  in  order  not  to  encroach  upon  the 
unsafe  operating  window.  A  CBM  life  cycle  model  was  incorporated  within  the  analytic 
architecture  for  a  cost-benefit  analysis  of  an  ISHM-enabled  UAS  as  compared  to  a  non- 

ISHM  UAS  requiring  scheduled  (or  time-based)  maintenance. 
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The  answers  to  the  literature  review  provided  the  foundation  in  the  development  of  the 
analytic  architecture  and  the  discrete-event  simulation  model.  Thereafter,  the  results  from 
the  simulation  model  serve  to  answer  the  following  research  questions: 

(1)  What  are  the  performance  characteristics  of  ISHM  to  ensure  continued 
airworthiness  of  the  UAS? 

The  Arena  ®  discrete-simulation  model  made  provision  for  investigation  of  various  UAS 
parameters  for  ISHM  implementation  based  on  the  analytic  architecture.  The  simulation 
model  defines  the  airworthiness  of  an  ISHM-enabled  UAS  through  the  output  parameter 
of  UAS  Expected  Lifetime.  Within  the  limits  of  defined  boundary  conditions,  it  was 
observed  that  the  failure  occurrence  rate  has  the  greatest  effect  on  the  UAS  expected 
lifetime.  For  an  extremely  low  failure  occurrence  rate  of  1  O'5,  it  is  possible  to  achieve 
over  a  million  flight  hours  for  an  UAS  with  or  without  ISHM  implementation.  This  result 
made  possible  the  tightening  of  experimental  boundaries  to  only  consider  scenarios  with 
typical  UAS  lifetimes  within  reasonable  failure  occurrence  rates. 

The  secondary  observation  from  the  simulation  model  is  that  for  a  non-ISHM  UAS,  the 
expected  lifetime  is  highly  sensitive  to  its  baseline  sensor  probability  of  detection. 
However,  for  an  ISHM-enabled  UAS,  in  scenarios  where  the  probability  of  ISHM 
confidence  is  higher  than  the  baseline  sensor  probability  of  detection,  the  expected 
lifetime  is  relatively  insensitive  to  the  latter,  but  highly  sensitive  to  the  former.  This  is 
explained  by  the  fact  that  for  an  ISHM-enabled  UAS,  the  eventual  declaration  of  a  failure 
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(or  non-failure)  from  its  diagnostics  and  prognostics  algorithms  will  override  the 
detection  results  by  the  baseline  sensor. 

Establishing  Performance  Requirements  for  an  ISHM-Enabled  UAS 

Initial  Airworthiness  -  In  the  preliminary  or  conceptual  design  stage  of  an  UAS,  the 
designer  should  be  aware  of  the  design  requirements,  i.e.  in  terms  of  the  desired  UAS 
expected  operational  lifetime  (output  performance).  From  the  results  of  the  simulation,  an 
associated  reliability  requirement  of  the  UAS  in  tenns  of  failure  rate  can  be  obtained  as  a 
perfonnance  specification.  This  overall  reliability  requirement  can  then  be  cascaded 
through  the  subsystems  of  the  UAS  based  on  reliability  allocation  design.  Secondly, 
perfonnance  specifications  in  terms  of  baseline  sensor  qualities  and  ISHM 
diagnostics/prognostics  capabilities  can  be  specified  by  the  parameters  of  Probability  of 
Sensor  Detection  and  Probability  of  ISHM  Confidence  respectively. 

Continued  Airworthiness  -  For  a  non-ISHM  UAS  in  the  operational  phase  of  its  life  cycle 
considering  an  ISHM  upgrade,  existing  perfonnance  data  on  failure  occurrence  rates  and 
baseline  sensor  detection  rates  would  have  been  available.  Comparing  the  available  field 
infonnation  with  the  simulation  results  will  assist  the  designer  in  defining  perfonnance 
specifications  in  terms  of  ISHM  diagnostics/prognostics  capabilities  to  achieve  a  desired 
reduction  in  maintenance  costs. 
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(2)  How  will  ISHM  provide  a  business  case  to  improve  the  level  of  UAS  self¬ 
autonomy? 

This  research  effort  developed  two  separate  Arena  ®  simulation  models  for  an  ISHM- 
enabled  UAS  and  non-ISHM  UAS.  Among  the  48  scenarios  tested,  only  26  scenarios 
showed  an  expected  gain  in  UAS  life  expectancy  with  ISHM  implementation.  Hence, 
ISHM  does  not  necessarily  improve  the  life  expectancy  of  a  UAS,  especially  in 
conditions  where  the  baseline  sensors  have  shown  excellent  performance  in  detection  of 
failure  conditions.  Therefore,  implementing  an  ISHM  system  with  poor  diagnostics 
and/or  prognostics  capabilities  will  only  do  more  hann  than  good.  We  conclude  that  in 
order  to  achieve  lifetime  extension  with  ISHM  implementation,  the  probability  of  ISHM 
confidence  must  be  greater  than  the  probability  of  detection  for  the  baseline  sensor. 

The  simulation  models  were  set-up  to  investigate  the  cost-effectiveness  of  a  CBM  policy 

for  an  ISHM-enabled  UAS  that  was  assumed  not  to  require  scheduled  maintenance  unlike 

the  non-ISHM  UAS.  Among  the  75  scenarios  tested  for  reduction  in  maintenance  actions, 

only  46  scenarios  showed  a  reduction  in  the  total  number  of  maintenance  actions  after 

ISHM  implementation.  This  result  cautioned  that  it  will  be  important  to  define  ISHM 

performance  specifications  in  terms  of  missed  detection  and  false  alarm  rates  so  that 

implementing  ISHM  onboard  a  UAS  will  not  induce  unnecessary  maintenance  actions.  In 

addition,  different  costs  components  attributed  to  the  various  types  of  maintenance 

actions  will  need  to  be  considered  to  weigh  against  the  implementation  cost  of  ISHM.  In 

summary,  the  simulation  results  from  the  test  scenarios  will  be  able  to  detennine  the 

performance  specifications  of  the  envisaged  ISHM  system,  by  considering  the  desired 
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benefits  of  ISHM  implementation  based  on  the  existing  reliability  of  the  UAS  and  the 
baseline  sensor  quality. 

(3)  What  are  the  potential  impacts  of  ISHM  to  maintenance  practices  and  life  cycle 
costs? 

One  of  the  perfonnance  measures  recorded  from  the  simulation  models  was  the  number 

of  Cannot-Duplicate  (CND)  cases,  where  a  failure  was  reported  in  flight  but  the  condition 

was  unable  to  be  duplicated  on  ground.  Not  only  does  CND  drive  additional  maintenance 

effort  in  terms  of  extended  troubleshooting  along  all  possible  paths  of  a  fault  tree 

analysis,  it  also  reduces  confidence  in  the  UAS  being  able  to  perfonn  subsequent 

missions  -  and  in  many  instances,  confidence  drives  changes  in  maintenance  policies.  A 

good  ISHM  system  should  reduce  both  the  number  of  missed  detections  and  false  alanns 

of  a  UAS,  and  minimizing  the  latter  will  mean  less  CND  cases.  It  is  expected  that  with 

increased  sophistication  of  future  UAS,  its  cost  will  also  increase  substantially.  This  may 

dictate  conservative  approaches  in  maintenance  of  such  high-value  assets.  In  releasing 

UAS  for  flight  with  a  CND,  replacement  of  components  for  precautionary  measure  could 

be  mandatory  -  inducing  higher  costs  for  UAS  maintenance.  A  desired  ISHM  system 

should  hence  act  as  a  safeguard  for  missed  detections,  while  being  a  filter  for  false 

alarms.  With  added  confidence  in  onboard  autonomy  through  ISHM,  it  will  then  be 

possible  to  relax  scheduled  maintenance  requirements  and,  in  the  long-term,  adopt  a 

condition-based  maintenance  (CBM)  program  with  potential  life  cycle  cost  savings.  It 

should  be  cautioned,  though,  that  CBM  drives  undesired  changes  to  maintenance  policies 

in  tenns  of  maintenance  scheduling.  CBM  will  be  more  difficult  to  forecast  and  plan  for 
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as  compared  to  time-based  maintenance,  and  may  result  in  inefficiencies  with  regard  to 
resource  allocation. 

5.3  Recommendations  for  Future  Research 

This  research  effort  provides  potential  designers  with  a  top-level  reference  in  detennining 
perfonnance  specifications  for  an  ISHM-enabled  UAS  to  achieve  desired  airworthiness 
and  maintenance  outcomes,  and  there  exists  potential  areas  of  future  research.  The 
Arena®  simulation  models  were  based  upon  the  analytic  architecture  and  it  would  be 
advisable  to  review  the  architecture  for  relevance  of  applications  in  future  research.  An 
important  emphasis  of  this  research  lie  in  the  accurate  documentation  of  model  input 
parameters,  assumptions,  implementation  and  results.  Documentation  is  important  to 
caveat  the  research  conclusions  within  tested  boundary  conditions,  and  provide  easy 
reference  for  potential  researchers  interested  in  this  work. 

Although  the  model  provides  provision  for  variation  in  many  parameters  (such  as  latency 
threshold,  defined  scheduled  maintenance  interval,  etc.),  not  all  of  them  were  assigned  as 
input/design  variables  in  the  analysis.  One  area  of  future  research  can  be  in  terms  of  an 
extensive  Design  of  Experiment  (DOE)  effort,  to  investigate  the  interactions  among  the 
various  variables  considered  together  to  achieve  a  desired  response.  Thereafter,  statistical 
regression  techniques  can  be  used  to  derive  a  model  equation  considering  the  input 
variables  with  significant  effects. 
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All  of  this  research  effort  used  theoretical  values  when  evaluating  the  simulation  model. 
The  levels  considered  for  the  test  scenarios  were  arbitrary  and  may  not  be  representative 
of  real-world  systems.  Instead  of  the  constant  failure  rate  model  that  is  currently 
assumed,  realistic  failure  sub-models  exhibiting  wear-out  conditions  can  be  developed  if 
failure  data  for  existing  UAS  are  made  available.  Similarly,  sensor  degradation  can  be 
incorporated  in  future  research  to  model  increased  sensor  maintenance  in  accordance  to 
actual  degradation  patterns  of  a  fielded  sensor.  In  addition,  when  infonnation  on  ISHM 
system  prototypes  are  available,  actual  performance  data  can  then  be  input  into  the 
simulation  model  to  yield  more  realistic  results. 

An  important  assumption  of  the  existing  architecture  and  discrete-event  simulation  model 
is  that  in  an  ISHM-enabled  UAS,  the  detection  outcome  by  the  baseline  sensor  and  the 
eventual  declaration  by  ISHM  were  assumed  to  be  independent  events,  with  the  latter 
being  the  final  authority  in  failure  declaration.  This  effectively  rendered  the  baseline 
sensor  as  a  ‘non-factor  ’  in  the  ISHM-enabled  UAS  simulation  model  in  determining  the 
expected  lifetime  of  the  UAS.  This  research  does  not  investigate  dependency  of  detection 
events,  or  different  diagnostics  or  prognostics  algorithms  in  resolving  conflicts  of 
baseline  sensor  and  ISHM  decisions,  and  future  research  could  expound  on  developing 
ISHM  algorithms  to  investigate  dependency  between  them  in  order  to  define  ISHM 
performance  characteristics. 
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5.4  Summary 

This  research  effort  developed  an  analytic  architecture  and  an  associated  discrete-event 


simulation  to  investigate  the  potential  benefits  of  ISHM  implementation  onboard  an 
UAS.  From  the  results  of  the  simulation,  it  was  shown  that  ISHM  presented  the  most 
cost-effective  improvement  over  baseline  systems  in  situations  where  the  reliability  of  the 
UAS  is  poor  (relative  to  manned  systems)  and  the  baseline  sensor  exhibited  poor  qualities 
in  terms  of  missed  detection  and  false  alarm  rates.  Through  simulation  experiments 
involving  defined  test  scenarios,  it  was  observed  that  failure  occurrence  rates,  sensor 
quality  characteristics  and  ISHM  perfonnance  specifications  were  significant  factors  in 
determining  the  output  responses  of  the  model.  Although  ISHM  presented  benefits  in  its 
envisioned  implementation,  the  test  scenarios  exhibited  instances  whereby  poor 
performance  specifications  of  ISHM  systems  would  lead  to  a  reduced  life  expectancy,  or 
increased  maintenance  actions,  especially  in  the  case  of  a  highly  reliable  UAS. 

It  is  important  to  note  that  the  results  of  this  research  seek  to  provide  potential  designers 
with  top-level  performance  specifications  of  an  ISHM  implementation.  However,  the 
results  of  the  analysis  are  only  bounded  by  the  defined  assumptions  of  the  simulation 
model.  The  analytic  architecture  is  only  a  piece  of  the  ISHM  puzzle,  and  should  be 
considered  with  other  analyses  to  achieve  the  desired  outcome  of  Integrated  System 
Health  Management  for  autonomous  UAS. 
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Appendix  A:  Description  of  Arena  ®  Modules  and  Sub-Models 


Initialization  Modules 


Title: 

Average  Sortie  Duration 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Assign  Module 

Diagram: 

f - 'I 

Average  Sortie 
Duration 

Purpose: 

Assigns  average  sortie  duration  as  a  parameter  to  the  simulation  model. 

Parameter: 

NormalASD  =  10  (hours) 

Assumption/ 

Additional 

Information: 

Assumed  that  an  UAS  mission  that  recovers  Code  1  or  Code  2  will  fly  an  average 
mission  length  of  10  hours. 

Title: 

Sortie  Generator 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Create  Module 

Diagram: 

Sortie  Generators 

Purpose: 

Creates  arrival  entities  (generate  sorties)  to  simulation  model  at  a  specified  rate. 

Parameter: 

Constant  inter-arrival  time  of  12  hours,  i.e.  one  sortie  generated  every  12  hours. 

Assumption/ 

Additional 

Information: 

Assumed  that  only  2  sorties  planned  in  a  24-hour  cycle  for  each  UAS. 

Title: 

Define  Scheduled  Maintenance  Interval 

Applicable  To: 

Non-ISHM  UAS  Model  Only 

Type: 

Assign  Module 

Diagram: 

define  Schedule^ 
Maintenance 
Interval 

>*u 

Purpose: 

Assigns  the  scheduled  maintenance  requirement  to  the  Non-ISHM  UAS  model. 

Parameter: 

DefinedMaintenancelnterval  =  1000  (hours) 

Assumption: 

Assumed  to  be  a  fixed  interval  throughout  the  life  of  the  UAS. 
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Title: 

Sortie  Probability  of  Failure 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Assign  Module 

Diagram: 

& - \ 

Sortie  Probability! 
of  Failure  | 

^  * 

Purpose: 

Calculates  probability  of  failure  for  particular  sortie  based  on  occurrence  rate. 

Parameter: 

FailureOccurrenceRate: 

Typical  Values  considered:  Iff1,  10'2,  1  O'3 (baseline),  10'4 ,  and  10 ~5 

Unit:  failure  per  hour. 

*  Based  on  suggested  probability  levels  defined  in  Table  2  / DoD ,  2000] . 

ProbFailure  =  1  _  e_Failure0ccurrenceRate  * NomalASD 

Variable  based  on  occurrence  rate. 

RandomFailureFlag:  UNIF(0,1,7) 

Assigns  the  first  common  random  number  stream  to  this  variable  as  a  means  of  variance 
reduction  between  the  ISHM-enabled  UAS  model  and  the  Non-ISHM  UAS  model. 

Assumption/ 

Additional 

Information: 

Failure  rate  assumed  to  be  relatively  constant  during  normal  UAS  operation  if  system 
design  is  mature  [Blanchard,  2004].  The  failure  rate  is  usually  what  is  presented  in 
FMECA  reports,  e.g.  1  failure  in  106  hours  (or  7  =  10"6  failure  per  hour).  Probability  of 
failure  is  assumed  to  follow  an  exponential  function  with  constant  failure  rate. 

Title: 

Failure  Occurred? 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Decide  Module 

Diagram: 

True 

■4^  Failure  Occurred?^* - 

Purpose: 

Determines  if  a  failure  condition  exists  for  each  sortie. 

Parameter: 

True  if  RandomFailureFlag  <=  ProbFailure 

Logic: 

Based  on  presence  (or  absence)  of  failure  condition  within  particular  sortie,  decides  on 
the  two  major  logic  flows  within  each  simulation  model. 

Assumption/ 

Additional 

Information: 

For  each  of  the  two  simulation  models,  there  exist  two  similar  major  logic  flows  that 
include  the  a)  failure  condition  present ,  and  b )  failure  condition  absent  flows. 
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Failure  Assessment  and  Detection  Sub-Models 


Title: _ 

Applicable  To: 
Type: 


A1.3  Perform  Failure  Assessment 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Sub-Model 


Purpose: 


Parameter: 


Determines  if  a  present  failure  condition  is  catastrophic  or  has  a  significant  accumulated 
latent  effect  from  prior  missed  detections. 


Number  of  Sorties  With  Failures:  Record  Module 
ProbCatastrophicFailure  =  26/132  =  0.197 
LatencyFactor  (range  from  0  to  5)  -  attribute  of  current  sortie 
AccumulatedLatencyEffect  -  system  variable  of  UAS 
RandomCatastrophicFlag:  UN1F(0,1,2) 

Assigns  the  second  common  random  number  stream  to  this  variable  as  a  means  of 
variance  reduction  between  the  ISHM-enabled  UAS  model  and  the  Non-ISHM  UAS 
model. 


Logic: 


Based  on  probability  of  a  failure  being  catastrophic,  assigns  a  status  flag  to  a  sortie.  Also 
checks  the  accumulated  latency  effect  from  missed  detections  of  past  sorties  against  the 
LatencyThreshold  (fixed  at  5)  to  ascertain  latency  factor  of  current  sortie. 


Assumption/ 

Additional 

Information: 


Based  on  NASA  report  in  Feb  2007,  there  are  26  potentially  catastrophic  failures  out  of 
132  identified  failure  conditions  for  a  generic  UAS  [Flayhurst  et  al,  2007]. 

Assumed  that  a  maximum  of  5  prior  missed  detections  will  result  in  the  upgrade  in 
severity  of  a  current  defect  that  will  result  in  a  mishap  if  undetected. 
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Title: 

A1.2  Perform  Failure  Detection 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Sub-Model 

Diagram: 

^  s 

Sensor  Detected  I 

.  Probability  of  1  a 

L£j 

■ 

Sensor  Detection  | 

- Failuro  Dctcctod?^^* - 

0 

r~ — \ 

Detection  Flan  | 

Purpose: 

Determines  if  a  present  failure  condition  is  detected  by  baseline  sensor. 

Parameter: 

ProbSensorDetection: 

Typical  Values  considered:  0.6,  0.7,  0.8  (baseline),  and  0.9 

RandomSensorDetectionFlag:  UNIF(0,1,J) 

Assigns  the  third  common  random  number  stream  to  this  variable  as  a  means  of 
variance  reduction  between  the  ISHM-enabled  UAS  model  and  the  Non-ISHM  UAS 
model. 

IF  RandomSensorDetectionFlag  <=  ProbSensorDetection, 

SensorDetected  (status  flag)  =  1,  ELSE  SensorDetected  (status  flag)  =  0. 

Logic: 

Based  on  probability  of  sensor  detection,  assigns  a  status  flag  to  a  sortie. 

Assumption/ 

Additional 

Information: 

Assumed  to  be  a  baseline  sensor  characteristic  for  generic  UAS.  Has  a  positive 
correlation  with  expected  UAS  lifetime. 
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Title: _ 

Applicable  To: 
Type: 


Diagram: 


A1.2  Perform  Failure  Detection  (False  Alarm) 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 
Sub-Model 


Purpose: 


Parameter: 


Determines  if  baseline  sensor  produces  a  false  alarm  when  no  failure  condition  exists. 


Number  of  Sorties  Without  Failure:  Record  Module 
ProbSensorFalseAlarm: 

Typical  Values  considered:  0.01,  0.02,  0.03  (baseline),  0.04  and  0.05 
RandomSensorFAFlag:  UNIF(0,1,4) 

Assigns  the  fourth  common  random  number  stream  to  this  variable  as  a  means  of 
variance  reduction  between  the  ISHM-enabled  UAS  model  and  the  Non-ISHM  UAS 
model. 

IF  RandomSensorFAFlag  <=  ProbSensorFalseAlarm, 

SensorFalseAlarm  (status  flag)  =  1,  ELSE  SensorFalseAlarm  (status  flag)  =  0. 


Logic: 


Based  on  probability  of  sensor  false  alarm,  assigns  a  status  flag  to  a  sortie. 


Assumption/ 

Additional 

Information: 


Assumed  to  be  a  baseline  sensor  characteristic  for  generic  UAS.  Has  a  positive 
correlation  with  unscheduled  sensor  maintenance  actions. 
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Title: 

A2.1  Perform  Diagnostics  and  Prognostics  Assessment 

Applicable  To: 

1SHM -Enabled  UAS  Model  Only 

Type: 

Sub-Model 

Diagram: 

|  L  I  ISHM  1 

^  Confirmation  Hag  I 

J  Probability  ol  1  K 

1  ISHM  Confidence  | 

ffciiure  DeterminedDWJ~__ 

ISHM? 

- ■  j!sHMMlSSHa^ 

Purpose: 

Determines  if  a  present  failure  condition  is  ascertained  by  ISHM. 

Parameter: 

ProbISHMConfidence : 

Typical  Values  considered:  0.6,  0.7,  0.8  (baseline),  and  0.9 

RandomISHMDetectionFlag:  UN1F(0,1,5) 

Assigns  the  fifth  common  random  number  stream  to  this  variable  as  a  means  of  variance 
reduction  between  the  ISHM-enabled  UAS  model  and  the  Non-ISHM  UAS  model. 

IF  RandomISHMDetectionFlag  <=  ProbISHMConfidence, 

ISHMConfirmed  (status  flag)  =  1,  ELSE  ISHMConfirmed  (status  flag)  =  0. 

Logic: 

Based  on  probability  of  ISHM  confidence  (representing  strength  of  ISHM  diagnostics 
and  prognostics  algorithms),  assigns  a  status  flag  to  a  sortie. 

Assumption/ 

Additional 

Information: 

Desired  performance  specification  of  ISHM  system.  Has  a  positive  correlation  with  the 
expected  UAS  lifetime. 
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Title: 

A2.1  Perform  Diagnostics  and  Prognostics  Assessment  (False  Alarm) 

Applicable  To: 

1SHM -Enabled  UAS  Model  Only 

Type: 

Sub-Model 

Diagram: 

1 

\ 

SI  IM  False  Atarml 

1 - — ISHM  False  Alarm^*^ - 

_ J  Probability  of  ^  | 

Flag 

- ■ 

ISHM  1  atse  Alai  rtf’ 

Purpose: 

Determines  if  1SFIM  produces  a  false  alarm  when  no  failure  condition  exists. 

Parameter: 

ProbISFIMFalseAlarm 

Typical  Values  considered:  0.01,  0.02,  0.03  (baseline),  0.04  and  0.05 

Random! SHMF AFlag :  UNIF(0,1,<5) 

Assigns  the  sixth  common  random  number  stream  to  this  variable  as  a  means  of 
variance  reduction  between  the  ISHM-enabled  UAS  model  and  the  Non-ISHM  UAS 
model, 

IF  RandomlSFIMF AFlag  <=  ProbISFIMFalseAlarm, 

ISHMFalseAlarm  (status  flag)  =  1,  ELSE  ISHMFalseAlarm  (status  flag)  =  0. 

Logic: 

Based  on  probability  of  ISHM  false  alarm  (representing  strength  of  ISHM  diagnostics 
and  prognostics  algorithms),  assigns  a  status  flag  to  a  sortie. 

Assumption/ 

Additional 

Information: 

Desired  performance  specification  of  ISHM  system.  Has  a  positive  correlation  with 
unscheduled  ISHM  maintenance  actions. 
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UAS  Recovery  States  Sub-Models 


Title: _ 

Applicable  To: 
Type: 


Diagram: 


Purpose: 


Parameter: 


UAS  Mishap  Determination 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 
Sub-Model 


Determines  if  a  present  failure  condition  will  result  in  a  mishap  if  undetected.  Terminate 
UAS  flight  upon  detection  of  first  mishap. 

LatencyThreshold  =  5 
UASCrashed  ( 1  or  0,  status  flag) 

Flight  Duration  to  Mishap  =  0.3  *  NormalASD  -  Implemented  as  a  process  delay  to 
calculate  the  accumulated  flight  time  of  a  UAS. 


Logic: 


Assumption/ 

Additional 

Information: 


Given  that  an  existing  failure  is  undetected,  an  UAS  mishap  will  occur  if  the  failure  is 
catastrophic,  or  if  the  UAS  had  5  previous  missed  detections  of  existing  failure 
condition. 

Assumed  that  a  maximum  of  5  prior  missed  detections  will  result  in  the  upgrade  in 
severity  of  a  current  defect  that  will  result  in  a  mishap  if  undetected. 

In  the  event  of  a  mishap,  it  is  assumed  that  the  UAS  will  fly  for  only  30%  of  its  normal 
sortie  duration. 


Title: _ 

Applicable  To: 
Type: 


Diagram: 


Purpose: 


Parameter: 


Logic: 

Assumption/ 

Additional 

Information: 


Code  3  Recoveries _ 

ISHM-Enabled  UAS  Model  Only 
Sub-Model 


Calculates  the  flight  time  of  a  Code  3  mission.  Also  add  an  unscheduled  component 
replacement  action. 

Code  3  Flight  Duration  =  0.5  *  NormalASD  -  Implemented  as  a  process  delay  to 
calculate  the  accumulated  flight  time  of  a  UAS. 

ComponentReplacement  -  Counter  for  number  of  component  replacements  being 
carried  out. 

Process  the  flight  duration  and  required  maintenance  activities  for  a  Code  3  recovery. 

In  the  event  of  a  Code  3,  it  is  assumed  that  the  UAS  will  fly  for  only  50%  of  its  normal 
sortie  duration. 
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Title: _ 

Applicable  To: 
Type: 


Diagram: 


Code  3  Recoveries _ 

Non-ISHM  UAS  Model  Only 
Sub-Model 


Code  .3  Reduced 

F  light  I  ime 

. 


Add  Unscheduled] 
Component 
Replacements 


n 


1  otal  Cole  3 
Unscheduled 
Component 
Replacement 


Reset 

Maintenance 


_ 


Purpose: 


Parameter: 


Calculates  the  flight  time  of  a  Code  3  mission.  Also  adds  an  unscheduled  component 
replacement  action  and  resets  the  maintenance  interval. 


Code  3  Flight  Duration  =  0.5  *  NormalASD  -  Implemented  as  a  process  delay  to 
calculate  the  accumulated  flight  time  of  a  UAS. 

ComponentReplacement  -  Counter  for  number  of  component  replacements  being 
carried  out. 

TimeToScheduledMaintenance  =  DefinedMaintenancelnterval 


Logic: 

Assumption/ 

Additional 

Information: 


Process  the  flight  duration  and  required  maintenance  activities  for  a  Code  3  recovery. 
Resets  the  scheduled  maintenance  interval  upon  completion  of  component  replacement. 

In  the  event  of  a  Code  3,  it  is  assumed  that  the  UAS  will  fly  for  only  50%  of  its  normal 
sortie  duration. 


Title: _ 

Applicable  To: 
Type: 


Diagram: 


Code  3  Recoveries  (CND) _ 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 
Sub-Model 


Code  3  CND 
Reduced  Flight 
Time 

_ r- 

DupScatc  CND  W 

f 

I- 

j 


Purpose: 


Parameter: 


Logic: 

Assumption/ 

Additional 

Information: 


Calculates  the  flight  time  of  a  Code  3  mission  due  to  ISHM  false  alarm  (for  the  ISHM- 
Enabled  UAS  model)  and  due  to  sensor  false  alarm  (for  the  Non-ISHM  UAS  model). 
Also  add  a  Cannot  Duplicate  (CND)  incident. 

Code  3  Flight  Duration  =  0.5  *  NormalASD  -  Implemented  as  a  process  delay  to 
calculate  the  accumulated  flight  time  of  a  UAS. 

CND  -  Counter  for  number  of  CND  incidents. 

Process  the  flight  duration  and  increment  the  number  of  CND  incidents  for  a  Code  3 
CND  recovery. 

In  the  event  of  a  reported  Code  3  (although  it  is  a  false  alarm),  it  is  assumed  that  the 
UAS  will  fly  for  only  50%  of  its  normal  sortie  duration. 


104 


Title: 

Code  2  Recoveries  (Sensor  False  Alarm  Only) 

Applicable  To: 

1SHM -Enabled  UAS  Model  Only 

Type: 

Sub-Model  with  Single  Process  Module  Only 

Diagram: 

Code  2  Normal 

Flight  Time  With 

Sensor  False 

Alarm 

0 

Purpose: 

Calculates  the  flight  time  of  a  Code  2  mission.  For  this  case,  this  sub-model  only  applies 
to  a  sortie  without  a  failure  condition;  correctly  diagnosed  by  ISHM  but  sensor  produces 
a  false  alarm. 

Parameter: 

Code  2  Flight  Duration  =  NormalASD  -  Implemented  as  a  process  delay  to  calculate  the 
accumulated  flight  time  of  a  UAS. 

Logic: 

Process  the  flight  duration  Code  2  recovery.  Requirement  for  sensor  maintenance  due  to 
false  alarm  is  being  processed  by  the  proceeding  sub-model. 

Assumption/ 

Additional 

Information: 

In  the  event  of  a  Code  2,  it  is  assumed  that  the  UAS  will  still  complete  its  planned 
mission  flying  its  normal  sortie  duration. 

Title: 

Code  2  Recoveries  (ISHM  Missed  Detection  Only) 

Applicable  To: 

ISHM -Enabled  UAS  Model  Only 

Type: 

Sub-Model 

Diagram: 

/  Add  Code  2  \ 

.  Unscheduled  I _ 

Component  | 
Replacements 

Code  2  Normal 
Right  Time 

J '  Unscheduled  l 

j  Component  i 

Replacement  1 

0 

Purpose: 

Calculates  the  flight  time  of  a  Code  2  mission.  For  this  case,  there  is  a  failure  condition 
that  is  correctly  detected  by  the  baseline  sensor  but  ISHM  denies  the  failure  condition. 
Hence,  also  add  an  unscheduled  component  replacement  action  as  a  failure  condition 
exists. 

Parameter: 

Code  2  Flight  Duration  =  NormalASD  -  Implemented  as  a  process  delay  to  calculate  the 
accumulated  flight  time  of  a  UAS. 

ComponentReplacement  -  Counter  for  number  of  component  replacements  being 
carried  out. 

Logic: 

Process  the  flight  duration  and  required  maintenance  activities  for  a  Code  2  recovery. 
Requirement  for  ISHM  maintenance  due  to  missed  detection  is  being  processed  by  the 
proceeding  sub-model. 

Assumption/ 

Additional 

Information: 

In  the  event  of  a  Code  2,  it  is  assumed  that  the  UAS  will  still  complete  its  planned 
mission  flying  its  normal  sortie  duration. 
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Title: 

Code  1  Recoveries 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Sub-Model 

Diagram: 

Code  1  Normal 

^A13  3  Assign  \ 

Failure  Latency  l 

rsghlTime 

Add  Latency  ( 

.1  EfficTJ  ■ 

0 

Purpose: 

Calculates  the  flight  time  of  a  Code  1  mission.  Also  assigns  failure  latency  to  UAS 
system  due  to  a  missed  detection  (by  both  ISHM  and  sensor  for  an  ISHM-Enabled  UAS 
model,  and  by  sensor  for  a  Non-ISHM  UAS  model). 

Parameter: 

Code  1  Flight  Duration  =  NormalASD  -  Implemented  as  a  process  delay  to  calculate  the 
accumulated  flight  time  of  a  UAS. 

AccumulatedLatencyEffect  -  system  variable  of  UAS  (increase  by  1) 

Logic: 

Process  the  flight  duration  and  assign  failure  latency  for  a  Code  1  recovery  with  missed 
detections  by  both  ISHM  and  baseline  sensor. 

Assumption/ 

Additional 

Information: 

In  the  event  of  a  Code  1  recovery  with  a  missed  detection  of  a  failure  condition,  it  is 
assumed  that  the  UAS  will  still  complete  its  planned  mission  flying  its  normal  sortie 
duration. 

Title: 

Code  1  Recoveries  (Good  Flight) 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Sub-Model  with  Single  Process  Module  Only 

Diagram: 

. 

Code  1  Good 
Flight  Normal 
Flight  Time 

r 

1 

0 

Purpose: 

Calculates  the  flight  time  of  an  uneventful  Code  1  mission. 

Parameter: 

Code  1  Flight  Duration  =  NormalASD  -  Implemented  as  a  process  delay  to  calculate  the 
accumulated  flight  time  of  a  UAS. 

Logic: 

Process  the  flight  duration  for  an  uneventful  Code  1  recovery. 
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Maintenance  Activities  Sub-Models 


Title: 


A14.1  Perform  Scheduled  Maintenance 


Applicable  To: 


Non-ISHM  UAS  Model  Only 


Type: 


Sub-Model 


Diagram: 


Purpose: 


Parameter: 


Logic: 


Determines  if  scheduled  maintenance  is  due. 

TimeToScheduledMaintenance  -  Time  counter  that  counts  down  towards  due  time  for 
scheduled  maintenance. 

ScheduledMaintenance  -  Counter  for  number  of  scheduled  maintenance  actions  being 
carried  out. 

Performs  scheduled  maintenance  action  if  TimeToScheduledMaintenance  is  less  than  or 
equals  to  0.  Resets  the  scheduled  maintenance  interval  upon  completion  of  scheduled 
maintenance  action. 


Assumption/ 

Additional 

Information: 


Even  though  scheduled  maintenance  actions  are  being  carried  out,  it  is  assumed  that  the 
failure  rate  of  the  UAS  does  not  change,  i.e.  system  is  mature  in  its  operational  phase 
with  constant  failure  rate. 


Title: _ 

Applicable  To: 
Type: 


Diagram: 


A22.1  Perform  Unscheduled  ISHM  Maintenance  (1SHM  Missed  Detection) 

ISHM-Enabled  UAS  Model  Only 

Sub-Model 


Purpose: 


Parameter: 


Logic: 

Assumption/ 

Additional 

Information: 


Determines  if  unscheduled  ISHM  maintenance  is  required. 

ISHMMissThreshold  =  4 

ISHMMaintenance  -  Counter  for  number  of  unscheduled  ISHM  maintenance  actions 
being  carried  out. 

Performs  unscheduled  ISHM  maintenance  action  if  ISHM  missed  detection  threshold  is 
exceeded.  Resets  ISHM  missed  detection  count  upon  completion  of  ISHM  maintenance. 

Assumed  to  be  an  arbitrary  target  threshold  to  carry  out  ISHM  maintenance  when 
exceeded;  should  be  lower  than  assumed  accumulated  latency  effect  to  be  meaningful. 


107 


Title: _ 

Applicable  To: 
Type: 


Diagram: 


A22.1  Perform  Unscheduled  ISHM  Maintenance  (1SHM  False  Alarm) 

ISHM-Enabled  UAS  Model  Only 

Sub-Model 


Purpose: 


Parameter: 


Logic: 


Assumption/ 

Additional 

Information: 


Determines  if  unscheduled  ISHM  maintenance  is  required. 

ISHMFalseAlarmThreshold  =  10 

ISHMMaintenance  -  Counter  for  number  of  unscheduled  ISHM  maintenance  actions 
being  carried  out. 

Performs  unscheduled  ISHM  maintenance  action  if  ISHM  false  alarm  threshold  is 
exceeded.  Resets  ISHM  false  alarm  count  upon  completion  of  ISHM  maintenance. 

Assumed  to  be  an  arbitrary  target  threshold  to  carry  out  ISHM  maintenance  when 
exceeded;  should  be  higher  that  missed  detection  threshold  due  to  no  safety  of  flight 
concern. 


Title: _ 

Applicable  To: 
Type: 


Diagram: 


A14.3  Perform  Unscheduled  Sensor  Maintenance  (Sensor  Missed  Detection) 

ISHM-Enabled  UAS  Model  Only 

Sub-Model 


Purpose: 


Parameter: 


Logic: 


Assumption/ 

Additional 

Information: 


Determines  if  unscheduled  sensor  maintenance  is  required. 

SensorMissThreshold  =  4 

SensorMaintenance  -  Counter  for  number  of  unscheduled  sensor  maintenance  actions 
being  carried  out. 

Performs  unscheduled  sensor  maintenance  action  if  sensor  missed  detection  threshold  is 
exceeded.  Resets  sensor  missed  detection  count  upon  completion  of  sensor  maintenance. 

Assumed  to  be  an  arbitrary  target  threshold  to  carry  out  sensor  maintenance  when 
exceeded;  should  be  lower  than  assumed  accumulated  latency  effect  to  be  meaningful. 
Only  applicable  for  an  ISHM-Enabled  UAS  model  since  for  the  isolated  baseline  sensor 
of  a  non-ISHM  UAS,  the  latter  would  not  know  that  it  has  missed  a  detection  without 
secondary  detection  mechanisms  (such  as  onboard  ISHM). 
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Title: 

A14.3  Perform  Unscheduled  Sensor  Maintenance  (Sensor  False  Alarm) 

Applicable  To: 

Both  ISHM-Enabled  UAS  and  Non-ISHM  UAS  Models 

Type: 

Sub-Model 

Add  Sensor 


Purpose: 

Determines  if  unscheduled  sensor  maintenance  is  required. 

Parameter: 

SensorFalseAlarmThreshold  =10 

SensorMaintenance  -  Counter  for  number  of  unscheduled  sensor  maintenance  actions 
being  carried  out. 

Logic: 

Performs  unscheduled  sensor  maintenance  action  if  sensor  false  alarm  threshold  is 
exceeded.  Resets  sensor  false  alarm  count  upon  completion  of  sensor  maintenance. 

Assumption/ 

Additional 

Information: 

Assumed  to  be  an  arbitrary  target  threshold  to  carry  out  sensor  maintenance  when 
exceeded;  should  be  higher  that  missed  detection  threshold  due  to  no  safety  of  flight 
concern. 
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Statistics  Record  Modules 


Title: 

Statistics  Collection 

Type: 

Record  Modules 

Purpose: 

Records  parameters  of  interest  for  verification  and  analysis. 

For  Both  Simulation  Models 

UAS  Expected  Lifetime 

Total  Unscheduled  Component  Replacements 

Total  Sensor  Maintenance 

Number  of  Sorties  With  Failures 

Number  of  Sorties  Without  Failures 

Module  Names: 

For  ISHM-Enabled  UAS  Model  Onlv 

Code  2  With  1SHM  Miss 

Code  1  With  Both  1SHM  and  Sensor  Misses 

Code  3  With  Sensor  Miss 

Code  3  With  Both  1SHM  and  Sensor  Confirmation 

Code  3  With  Both  1SHM  and  Sensor  False  Alarms 

Code  3  With  1SHM  False  Alarm  Only 

Code  2  With  Sensor  False  Alarm 

Code  1  Good  Flight  Without  False  Alarm 

Total  ISHM  Maintenance 

For  Non-ISHM  UAS  Onlv 

Code  3  Sensor  Detected 

Code  1  Sensor  Miss 

Code  3  CND  With  Sensor  False  Alarm 

Code  1  Good  Flight  Without  False  Alarm 

Total  Scheduled  Maintenance 
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Appendix  B:  Analysis  Results  -  Experimental  Objectives  and  Ranking  Results 


Objective  1:  Investigate  Dependence  of  UAS  Expected  Lifetime  on  Failure  Rate,  k 

Table  B-l  -  Dependence  ofUAS  Expected  Lifetime  on  Failure  Rate,  X 


Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Flour] 

Probability  of 

Detection 

(Baseline  Sensor) 

Probability  of  ISHM 
Confidence 

UAS  Lifetime 
( ISHM-Enabl ed  UAS) 
[Flours] 

UAS  Lifetime 
(Non-ISHMUAS) 
[Flours] 

1 

0.00001 

0.6 

0.9 

3,662,213.50 

1,025,301.50 

2 

0.00001 

0.7 

0.9 

3,662,213.50 

1,490,861.00 

3 

0.00001 

0.8 

0.9 

3,662,213.50 

2,423,547.00 

4 

0.00001 

0.9 

0.9 

3,662,213.50 

7,199,106.50 

5 

0.00001 

0.6 

0.8 

2,029,693.00 

1,025,301.50 

6 

0.00001 

0.7 

0.8 

2,029,693.00 

1,490,861.00 

7 

0.00001 

0.8 

0.8 

2,029,693.00 

2,423,547.00 

8 

0.00001 

0.9 

0.8 

2,029,693.00 

7,199,106.50 

9 

0.00001 

0.6 

0.7 

1,722,959.50 

1,025,301.50 

10 

0.00001 

0.7 

0.7 

1,722,959.50 

1,490,861.00 

11 

0.00001 

0.8 

0.7 

1,722,959.50 

2,423,547.00 

12 

0.00001 

0.9 

0.7 

1,722,959.50 

7,199,106.50 

13 

0.00001 

0.6 

0.6 

1,541,751.50 

1,025,301.50 

14 

0.00001 

0.7 

0.6 

1,541,751.50 

1,490,861.00 

15 

0.00001 

0.8 

0.6 

1,541,751.50 

2,423,547.00 

16 

0.00001 

0.9 

0.6 

1,541,751.50 

7,199,106.50 

17 

0.0001 

0.6 

0.9 

354,312.50 

103,479.50 

18 

0.0001 

0.7 

0.9 

354,312.50 

154,323.50 

19 

0.0001 

0.8 

0.9 

354,312.50 

253,859.00 

20 

0.0001 

0.9 

0.9 

354,312.50 

731,245.50 

21 

0.0001 

0.6 

0.8 

191,844.50 

103,479.50 

22 

0.0001 

0.7 

0.8 

191,844.50 

154,323.50 

23 

0.0001 

0.8 

0.8 

191,844.50 

253,859.00 

24 

0.0001 

0.9 

0.8 

191,844.50 

731,245.50 

25 

0.0001 

0.6 

0.7 

158,851.50 

103,479.50 

26 

0.0001 

0.7 

0.7 

158,851.50 

154,323.50 

27 

0.0001 

0.8 

0.7 

158,851.50 

253,859.00 

28 

0.0001 

0.9 

0.7 

158,851.50 

731,245.50 

29 

0.0001 

0.6 

0.6 

139,169.00 

103,479.50 

30 

0.0001 

0.7 

0.6 

139,169.00 

154,323.50 

31 

0.0001 

0.8 

0.6 

139,169.00 

253,859.00 

32 

0.0001 

0.9 

0.6 

139,169.00 

731,245.50 

33 

0.001 

0.6 

0.9 

38,676.00 

10,755.50 

34 

0.001 

0.7 

0.9 

38,676.00 

15,623.00 

35 

0.001 

0.8 

0.9 

38,676.00 

24,979.50 

36 

0.001 

0.9 

0.9 

38,676.00 

74,975.50 

37 

0.001 

0.6 

0.8 

22,096.50 

10,755.50 

38 

0.001 

0.7 

0.8 

22,096.50 

15,623.00 

39 

0.001 

0.8 

0.8 

22,096.50 

24,979.50 

40 

0.001 

0.9 

0.8 

22,096.50 

74,975.50 
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Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of 

Detection 

(Baseline  Sensor) 

Probability  of  ISHM 
Confidence 

UAS  Lifetime 
(ISHM-Enabled  UAS) 
[Hours] 

UAS  Lifetime 
(Non-ISHM  UAS) 
[Hours] 

41 

0.001 

0.6 

0.7 

18,828.00 

10,755.50 

42 

0.001 

0.7 

0.7 

18,828.00 

15,623.00 

43 

0.001 

0.8 

0.7 

18,828.00 

24,979.50 

44 

0.001 

0.9 

0.7 

18,828.00 

74,975.50 

45 

0.001 

0.6 

0.6 

16,477.50 

10,755.50 

46 

0.001 

0.7 

0.6 

16,477.50 

15,623.00 

47 

0.001 

0.8 

0.6 

16,477.50 

24,979.50 

48 

0.001 

0.9 

0.6 

16,477.50 

74,975.50 

49 

0.01 

0.6 

0.9 

3,878.50 

1,105.00 

50 

0.01 

0.7 

0.9 

3,878.50 

1,687.50 

51 

0.01 

0.8 

0.9 

3,878.50 

2,634.50 

52 

0.01 

0.9 

0.9 

3,878.50 

7,679.50 

53 

0.01 

0.6 

0.8 

2,266.00 

1,105.00 

54 

0.01 

0.7 

0.8 

2,266.00 

1,687.50 

55 

0.01 

0.8 

0.8 

2,266.00 

2,634.50 

56 

0.01 

0.9 

0.8 

2,266.00 

7,679.50 

57 

0.01 

0.6 

0.7 

1,914.50 

1,105.00 

58 

0.01 

0.7 

0.7 

1,914.50 

1,687.50 

59 

0.01 

0.8 

0.7 

1,914.50 

2,634.50 

60 

0.01 

0.9 

0.7 

1,914.50 

7,679.50 

61 

0.01 

0.6 

0.6 

1,689.00 

1,105.00 

62 

0.01 

0.7 

0.6 

1,689.00 

1,687.50 

63 

0.01 

0.8 

0.6 

1,689.00 

2,634.50 

64 

0.01 

0.9 

0.6 

1,689.00 

7,679.50 

65 

0.1 

0.6 

0.9 

432.50 

121.00 

66 

0.1 

0.7 

0.9 

432.50 

189.00 

67 

0.1 

0.8 

0.9 

432.50 

307.50 

68 

0.1 

0.9 

0.9 

432.50 

865.50 

69 

0.1 

0.6 

0.8 

246.50 

121.00 

70 

0.1 

0.7 

0.8 

246.50 

189.00 

71 

0.1 

0.8 

0.8 

246.50 

307.50 

72 

0.1 

0.9 

0.8 

246.50 

865.50 

73 

0.1 

0.6 

0.7 

208.00 

121.00 

74 

0.1 

0.7 

0.7 

208.00 

189.00 

75 

0.1 

0.8 

0.7 

208.00 

307.50 

76 

0.1 

0.9 

0.7 

208.00 

865.50 

77 

0.1 

0.6 

0.6 

185.00 

121.00 

78 

0.1 

0.7 

0.6 

185.00 

189.00 

79 

0.1 

0.8 

0.6 

185.00 

307.50 

80 

0.1 

0.9 

0.6 

185.00 

865.50 
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Objective  2:  Maximize  Percentage  of  Lifetime  Extension  through  ISHM 
Implementation 


Table  B-2  -  Maximize  Percentage  of  Lifetime  Extension  through  ISHM  Implementation 


Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of 
Detection 

(Baseline  Sensor) 

Probability  of  ISHM 
Confidence 

UAS  Lifetime 
(ISHM-Enabled  UAS) 
[Hours] 

(a) 

UAS  Lifetime 
(Non-ISHM  UAS) 
[Hours] 

(b) 

Increase  in  UAS 

Lifetime 
(c)  =  (a)-(b) 

%  Lifetime  Gain 
(d)  =  (c)/(b) 

1 

0.001 

0.6 

0.9 

38676 

10755.5 

27920.5 

259.59% 

2 

0.01 

0.6 

0.9 

3878.5 

1105 

2773.5 

251.00% 

3 

0.0001 

0.6 

0.9 

354312.5 

103479.5 

250833 

242.40% 

4 

0.001 

0.7 

0.9 

38676 

15623 

23053 

147.56% 

5 

0.01 

0.7 

0.9 

3878.5 

1687.5 

2191 

129.84% 

6 

0.0001 

0.7 

0.9 

354312.5 

154323.5 

199989 

129.59% 

7 

0.001 

0.6 

0.8 

22096.5 

10755.5 

11341 

105.44% 

8 

0.01 

0.6 

0.8 

2266 

1105 

1161 

105.07% 

9 

0.0001 

0.6 

0.8 

191844.5 

103479.5 

88365 

85.39% 

10 

0.001 

0.6 

0.7 

18828 

10755.5 

8072.5 

75.05% 

11 

0.01 

0.6 

0.7 

1914.5 

1105 

809.5 

73.26% 

12 

0.001 

0.8 

0.9 

38676 

24979.5 

13696.5 

54.83% 

13 

0.0001 

0.6 

0.7 

158851.5 

103479.5 

55372 

53.51% 

14 

0.001 

0.6 

0.6 

16477.5 

10755.5 

5722 

53.20% 

15 

0.01 

0.6 

0.6 

1689 

1105 

584 

52.85% 

16 

0.01 

0.8 

0.9 

3878.5 

2634.5 

1244 

47.22% 

17 

0.001 

0.7 

0.8 

22096.5 

15623 

6473.5 

41.44% 

18 

0.0001 

0.8 

0.9 

354312.5 

253859 

100453.5 

39.57% 

19 

0.0001 

0.6 

0.6 

139169 

103479.5 

35689.5 

34.49% 

20 

0.01 

0.7 

0.8 

2266 

1687.5 

578.5 

34.28% 

21 

0.0001 

0.7 

0.8 

191844.5 

154323.5 

37521 

24.31% 

22 

0.001 

0.7 

0.7 

18828 

15623 

3205 

20.51% 

23 

0.01 

0.7 

0.7 

1914.5 

1687.5 

227 

13.45% 

24 

0.001 

0.7 

0.6 

16477.5 

15623 

854.5 

5.47% 

25 

0.0001 

0.7 

0.7 

158851.5 

154323.5 

4528 

2.93% 

26 

0.01 

0.7 

0.6 

1689 

1687.5 

1.5 

0.09% 

27 

0.0001 

0.7 

0.6 

139169 

154323.5 

-15154.5 

-9.82% 

28 

0.001 

0.8 

0.8 

22096.5 

24979.5 

-2883 

-11.54% 

29 

0.01 

0.8 

0.8 

2266 

2634.5 

-368.5 

-13.99% 

30 

0.0001 

0.8 

0.8 

191844.5 

253859 

-62014.5 

-24.43% 

31 

0.001 

0.8 

0.7 

18828 

24979.5 

-6151.5 

-24.63% 

32 

0.01 

0.8 

0.7 

1914.5 

2634.5 

-720 

-27.33% 

33 

0.001 

0.8 

0.6 

16477.5 

24979.5 

-8502 

-34.04% 

34 

0.01 

0.8 

0.6 

1689 

2634.5 

-945.5 

-35.89% 

35 

0.0001 

0.8 

0.7 

158851.5 

253859 

-95007.5 

-37.43% 

36 

0.0001 

0.8 

0.6 

139169 

253859 

-114690 

-45.18% 

37 

0.001 

0.9 

0.9 

38676 

74975.5 

-36299.5 

-48.42% 

38 

0.01 

0.9 

0.9 

3878.5 

7679.5 

-3801 

-49.50% 

39 

0.0001 

0.9 

0.9 

354312.5 

731245.5 

-376933 

-51.55% 

40 

0.01 

0.9 

0.8 

2266 

7679.5 

-5413.5 

-70.49% 

41 

0.001 

0.9 

0.8 

22096.5 

74975.5 

-52879 

-70.53% 

42 

0.0001 

0.9 

0.8 

191844.5 

731245.5 

-539401 

-73.76% 

43 

0.001 

0.9 

0.7 

18828 

74975.5 

-56147.5 

-74.89% 

44 

0.01 

0.9 

0.7 

1914.5 

7679.5 

-5765 

-75.07% 

45 

0.01 

0.9 

0.6 

1689 

7679.5 

-5990.5 

-78.01% 

46 

0.001 

0.9 

0.6 

16477.5 

74975.5 

-58498 

-78.02% 

47 

0.0001 

0.9 

0.7 

158851.5 

731245.5 

-572394 

-78.28% 

48 

0.0001 

0.9 

0.6 

139169 

731245.5 

-592076.5 

-80.97% 
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Objective  3:  Minimize  Number  of  Cannot  Duplicate  (CND)  Cases  through  ISHM 
Implementation 


Table  B-3  -  Minimize  Number  of  Cannot  Duplicate  (CND)  Cases  through  ISHM  Implementation 


Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of  False 
Alarm 

(Baseline  Sensor) 

Probability  of 
ISHM  False  Alarm 

Total  Cases  of  CND 
(ISHM-Enabled  UAS) 
[a] 

Total  Cases  of  CND 
(Non-ISHM  UAS) 

[b] 

Reduction  in  CND 
[c]=[b]-[a] 

%  Reduction  in 

CND 

[d]=[c]/[b] 

1 

0.0001 

0.05 

0.01 

193 

1286 

1093 

84.99% 

2 

0.0001 

0.05 

0.02 

390 

1286 

896 

69.67% 

3 

0.0001 

0.04 

0.01 

193 

1023 

830 

81.13% 

4 

0.0001 

0.05 

0.03 

584 

1286 

702 

54.59% 

5 

0.0001 

0.04 

0.02 

390 

1023 

633 

61.88% 

6 

0.0001 

0.03 

0.01 

193 

768 

575 

74.87% 

7 

0.0001 

0.05 

0.04 

778 

1286 

508 

39.50% 

8 

0.0001 

0.04 

0.03 

584 

1023 

439 

42.91% 

9 

0.0001 

0.03 

0.02 

390 

768 

378 

49.22% 

10 

0.0001 

0.02 

0.01 

193 

515 

322 

62.52% 

11 

0.0001 

0.05 

0.05 

977 

1286 

309 

24.03% 

12 

0.0001 

0.04 

0.04 

778 

1023 

245 

23.95% 

13 

0.0001 

0.03 

0.03 

584 

768 

184 

23.96% 

14 

0.0001 

0.02 

0.02 

390 

515 

125 

24.27% 

15 

0.001 

0.05 

0.01 

22 

128 

106 

82.81% 

16 

0.001 

0.05 

0.02 

45 

128 

83 

64.84% 

17 

0.001 

0.04 

0.01 

22 

103 

81 

78.64% 

18 

0.001 

0.05 

0.03 

66 

128 

62 

48.44% 

19 

0.0001 

0.01 

0.01 

193 

252 

59 

23.41% 

20 

0.001 

0.04 

0.02 

45 

103 

58 

56.31% 

21 

0.001 

0.03 

0.01 

22 

76 

54 

71.05% 

22 

0.0001 

0.04 

0.05 

977 

1023 

46 

4.50% 

23 

0.001 

0.05 

0.04 

89 

128 

39 

30.47% 

24 

0.001 

0.04 

0.03 

66 

103 

37 

35.92% 

25 

0.001 

0.03 

0.02 

45 

76 

31 

40.79% 

26 

0.001 

0.02 

0.01 

22 

51 

29 

56.86% 

27 

0.001 

0.05 

0.05 

111 

128 

17 

13.28% 

28 

0.001 

0.04 

0.04 

89 

103 

14 

13.59% 

29 

0.01 

0.05 

0.01 

2 

13 

11 

84.62% 

30 

0.001 

0.03 

0.03 

66 

76 

10 

13.16% 

31 

0.01 

0.04 

0.01 

2 

10 

8 

80.00% 

32 

0.01 

0.05 

0.02 

5 

13 

8 

61.54% 

33 

0.01 

0.03 

0.01 

2 

8 

6 

75.00% 

34 

0.01 

0.05 

0.03 

7 

13 

6 

46.15% 

35 

0.001 

0.02 

0.02 

45 

51 

6 

11.76% 

36 

0.01 

0.04 

0.02 

5 

10 

5 

50.00% 

37 

0.01 

0.05 

0.04 

9 

13 

4 

30.77% 

38 

0.01 

0.02 

0.01 

2 

5 

3 

60.00% 

39 

0.01 

0.03 

0.02 

5 

8 

3 

37.50% 

40 

0.01 

0.04 

0.03 

7 

10 

3 

30.00% 

41 

0.001 

0.01 

0.01 

22 

25 

3 

12.00% 

42 

0.01 

0.05 

0.05 

11 

13 

2 

15.38% 

43 

0.01 

0.03 

0.03 

7 

8 

1 

12.50% 

44 

0.01 

0.04 

0.04 

9 

10 

1 

10.00% 

45 

0.01 

0.01 

0.01 

2 

2 

0 

0.00% 

46 

0.01 

0.02 

0.02 

5 

5 

0 

0.00% 

47 

0.01 

0.04 

0.05 

11 

10 

-1 

-10.00% 

48 

0.01 

0.03 

0.04 

9 

8 

-1 

-12.50% 

49 

0.01 

0.02 

0.03 

7 

5 

-2 

-40.00% 

50 

0.01 

0.03 

0.05 

11 

8 

-3 

-37.50% 
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Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of  False 
Alarm 

(Baseline  Sensor) 

Probability  of 
ISHM  False  Alarm 

Total  Cases  of  CND 
(ISHM-Enabled  UAS) 
[a] 

Total  Cases  of  CND 
(Non-ISHM  UAS) 

[b] 

Reduction  in  CND 
[c]=[b]-[a] 

%  Reduction  in 

CND 

[d]=[c]/[b] 

51 

0.01 

0.01 

0.02 

5 

2 

-3 

-150.00% 

52 

0.01 

0.02 

0.04 

9 

5 

-4 

-80.00% 

53 

0.01 

0.01 

0.03 

7 

2 

-5 

-250.00% 

54 

0.01 

0.02 

0.05 

11 

5 

-6 

-120.00% 

55 

0.01 

0.01 

0.04 

9 

2 

-7 

-350.00% 

56 

0.001 

0.04 

0.05 

111 

103 

-8 

-7.77% 

57 

0.01 

0.01 

0.05 

11 

2 

-9 

-450.00% 

58 

0.0001 

0.03 

0.04 

778 

768 

-10 

-1.30% 

59 

0.001 

0.03 

0.04 

89 

76 

-13 

-17.11% 

60 

0.001 

0.02 

0.03 

66 

51 

-15 

-29.41% 

61 

0.001 

0.01 

0.02 

45 

25 

-20 

-80.00% 

62 

0.001 

0.03 

0.05 

111 

76 

-35 

-46.05% 

63 

0.001 

0.02 

0.04 

89 

51 

-38 

-74.51% 

64 

0.001 

0.01 

0.03 

66 

25 

-41 

-164.00% 

65 

0.001 

0.02 

0.05 

111 

51 

-60 

-117.65% 

66 

0.001 

0.01 

0.04 

89 

25 

-64 

-256.00% 

67 

0.0001 

0.02 

0.03 

584 

515 

-69 

-13.40% 

68 

0.001 

0.01 

0.05 

111 

25 

-86 

-344.00% 

69 

0.0001 

0.01 

0.02 

390 

252 

-138 

-54.76% 

70 

0.0001 

0.03 

0.05 

977 

768 

-209 

-27.21% 

71 

0.0001 

0.02 

0.04 

778 

515 

-263 

-51.07% 

72 

0.0001 

0.01 

0.03 

584 

252 

-332 

-131.75% 

73 

0.0001 

0.02 

0.05 

977 

515 

-462 

-89.71% 

74 

0.0001 

0.01 

0.04 

778 

252 

-526 

-208.73% 

75 

0.0001 

0.01 

0.05 

977 

252 

-725 

-287.70% 
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Objective  4:  Minimize  Number  of  Maintenance  Actions  through  ISHM 
Implementation 


Table  B-4  -  Minimize  Number  of  Maintenance  Actions  through  ISHM  Implementation 


Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of  False 

Alarm 

(Baseline  Sensor) 

Probability  of 
ISHM  False  Alarm 

Total  Maintenance 

Actions 

(ISHM-Enabled  UAS) 
[a] 

Total  Maintenance 

Actions 

(Non-ISHM  UAS) 

[b] 

Reduction  of 

Maintenance 

Actions 

[c]=[b]-[a] 

%  Reduction  in 

Maintenance 

Actions 

[d]=[c]/[b] 

1 

0.0001 

0.05 

0.01 

309 

1655 

1346 

81.33% 

2 

0.0001 

0.05 

0.02 

526 

1655 

1129 

68.22% 

3 

0.0001 

0.04 

0.01 

289 

1367 

1078 

78.86% 

4 

0.0001 

0.05 

0.03 

739 

1655 

916 

55.35% 

5 

0.0001 

0.04 

0.02 

506 

1367 

861 

62.98% 

6 

0.0001 

0.03 

0.01 

270 

1088 

818 

75.18% 

7 

0.0001 

0.05 

0.04 

952 

1655 

703 

42.48% 

8 

0.0001 

0.04 

0.03 

719 

1367 

648 

47.40% 

9 

0.0001 

0.03 

0.02 

487 

1088 

601 

55.24% 

10 

0.0001 

0.02 

0.01 

252 

811 

559 

68.93% 

11 

0.0001 

0.05 

0.05 

1171 

1655 

484 

29.24% 

12 

0.0001 

0.04 

0.04 

932 

1367 

435 

31.82% 

13 

0.0001 

0.03 

0.03 

700 

1088 

388 

35.66% 

14 

0.0001 

0.02 

0.02 

469 

811 

342 

42.17% 

15 

0.0001 

0.01 

0.01 

232 

523 

291 

55.64% 

16 

0.0001 

0.04 

0.05 

1151 

1367 

216 

15.80% 

17 

0.0001 

0.03 

0.04 

913 

1088 

175 

16.08% 

18 

0.0001 

0.02 

0.03 

682 

811 

129 

15.91% 

19 

0.001 

0.05 

0.01 

36 

157 

121 

77.13% 

20 

0.001 

0.05 

0.02 

61 

157 

96 

61.25% 

21 

0.001 

0.04 

0.01 

34 

130 

96 

73.85% 

22 

0.0001 

0.01 

0.02 

449 

523 

74 

14.15% 

23 

0.001 

0.05 

0.03 

84 

157 

73 

46.63% 

24 

0.001 

0.04 

0.02 

59 

130 

71 

54.62% 

25 

0.001 

0.03 

0.01 

32 

100 

68 

68.00% 

26 

0.001 

0.05 

0.04 

109 

157 

48 

30.75% 

27 

0.001 

0.04 

0.03 

82 

130 

48 

36.92% 

28 

0.001 

0.02 

0.01 

29 

73 

44 

60.27% 

29 

0.001 

0.03 

0.02 

57 

100 

43 

43.00% 

30 

0.001 

0.05 

0.05 

134 

157 

23 

14.87% 

31 

0.001 

0.04 

0.04 

107 

130 

23 

17.69% 

32 

0.001 

0.03 

0.03 

80 

100 

20 

20.00% 

33 

0.001 

0.02 

0.02 

54 

73 

19 

26.03% 

34 

0.001 

0.01 

0.01 

27 

44 

17 

38.64% 

35 

0.01 

0.05 

0.01 

4 

15 

11 

73.15% 

36 

0.01 

0.04 

0.01 

4 

12 

8 

66.67% 

37 

0.01 

0.05 

0.02 

7 

15 

8 

53.02% 

38 

0.01 

0.03 

0.01 

3 

10 

7 

70.00% 

39 

0.01 

0.05 

0.03 

9 

15 

6 

39.60% 

40 

0.01 

0.04 

0.02 

7 

12 

5 

41.67% 

41 

0.01 

0.03 

0.02 

6 

10 

4 

40.00% 

42 

0.01 

0.02 

0.01 

3 

6 

3 

50.00% 

43 

0.01 

0.04 

0.03 

9 

12 

3 

25.00% 

44 

0.01 

0.05 

0.04 

12 

15 

3 

19.46% 

45 

0.01 

0.03 

0.03 

8 

10 

2 

20.00% 

46 

0.01 

0.05 

0.05 

14 

15 

1 

6.04% 

47 

0.01 

0.01 

0.01 

3 

3 

0 

0.00% 

48 

0.01 

0.02 

0.02 

6 

6 

0 

0.00% 

49 

0.01 

0.04 

0.04 

12 

12 

0 

0.00% 

50 

0.01 

0.03 

0.04 

11 

10 

-1 

-10.00% 
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Ranking 

Failure  Occurrence 

Rate 

[Per  Flying  Hour] 

Probability  of  False 
Alarm 

(Baseline  Sensor) 

Probability  of 
ISHM  False  Alarm 

Total  Maintenance 

Actions 

(ISHM-Enabled  UAS) 
[a] 

Total  Maintenance 

Actions 

(Non-ISHM  UAS) 

[b] 

Reduction  of 

Maintenance 

Actions 

[c]=[b]-[a] 

%  Reduction  in 

Maintenance 

Actions 

[d]=[c]/[b] 

51 

0.001 

0.04 

0.05 

132 

130 

-2 

-1.54% 

52 

0.01 

0.04 

0.05 

14 

12 

-2 

-16.67% 

53 

0.01 

0.02 

0.03 

8 

6 

-2 

-33.33% 

54 

0.01 

0.03 

0.05 

13 

10 

-3 

-30.00% 

55 

0.01 

0.01 

0.02 

6 

3 

-3 

-100.00% 

56 

0.001 

0.02 

0.03 

77 

73 

-4 

-5.48% 

57 

0.001 

0.03 

0.04 

105 

100 

-5 

-5.00% 

58 

0.01 

0.02 

0.04 

11 

6 

-5 

-83.33% 

59 

0.01 

0.01 

0.03 

8 

3 

-5 

-166.67% 

60 

0.01 

0.02 

0.05 

13 

6 

-7 

-116.67% 

61 

0.001 

0.01 

0.02 

52 

44 

-8 

-18.18% 

62 

0.01 

0.01 

0.04 

11 

3 

-8 

-266.67% 

63 

0.01 

0.01 

0.05 

13 

3 

-10 

-333.33% 

64 

0.001 

0.02 

0.04 

102 

73 

-29 

-39.73% 

65 

0.001 

0.03 

0.05 

130 

100 

-30 

-30.00% 

66 

0.001 

0.01 

0.03 

75 

44 

-31 

-70.45% 

67 

0.0001 

0.03 

0.05 

1132 

1088 

-44 

-4.04% 

68 

0.001 

0.02 

0.05 

127 

73 

-54 

-73.97% 

69 

0.001 

0.01 

0.04 

100 

44 

-56 

-127.27% 

70 

0.001 

0.01 

0.05 

125 

44 

-81 

-184.09% 

71 

0.0001 

0.02 

0.04 

895 

811 

-84 

-10.36% 

72 

0.0001 

0.01 

0.03 

662 

523 

-139 

-26.58% 

73 

0.0001 

0.02 

0.05 

1114 

811 

-303 

-37.36% 

74 

0.0001 

0.01 

0.04 

875 

523 

-352 

-67.30% 

75 

0.0001 

0.01 

0.05 

1094 

523 

-571 

-109.18% 
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